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Executive  Summary 

Proposition  E-The  Muni  Reform  Initiative 

On  November  2,  1999,  the  voters  of  San  Francisco  overwhelmingly 
approved  Proposition  E.  the  most  substantial  reformation  in  Muni 
history.  The  voters'  intent  was  to  institute  structural,  administrative 
and  financial  reforms  designed  to  provide  Muni  with  the  “resources, 
independence  and  focus  necessary"  to  become  one  of  the  best  urban 
transit  systems  in  the  world.  Recognizing  the  City’s  dependence  on 
public  transit  and  its  need  for  efficient  and  reliable  transit  service 
that  can  compete  with  the  private  automobile,  the  drafters  of  the 
initiative  sought  to  restructure  the  City’s  provision  and  administration 
of  transportation  and  parking  services  and  strengthen  the  City's 
Transit-First  Policy. 

The  overall  goals  for  transit  service  articulated  in  Proposition  E 
(now  Article  VII I A of  the  San  Francisco  City  Charter)  are  as  follows 
(Settion  8A.  100): 

1.  Reliable,  safe,  timely,  frequent,  and  convenient  service  to  all 
neighborhoods; 

2.  A reduction  in  breakdowns, delays. over-crowding,preventable 
accidents; 

3.  Clean  and  comfortable  vehicles  and  stations,  operated  by 
competent,  courteous,  and  well-trained  employees; 

4.  Support  and  accommodation  of  the  special  transportation 
needs  of  the  elderly  and  the  disabled; 

5.  Protection  from  crime  and  inappropriate  passenger  behavior 
on  the  Municipal  Railway;  and 

6.  Responsive,  efficient  and  accountable  management 
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To  achieve  these  goals.  Article  VIIIA  created  the  Municipal 
Transportation  Agency  (MTA),  combining  the  responsibility  for  street 
operation  (Department  of  Parking  and  Traffic)  with  the  dominant 
user  of  the  streets  — Muni.  Article  VIIIA  also  established  service 
standards  and  accountability  measures,  and  required  an  independent 
biennial  quality  review  of  transit  operations.  This  report  represents 
the  findings  of  the  first  independent  review  of  Muni's  performance 
since  the  passage  of  Proposition  E. 
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Proposition  £ SerWce  Standards 

The  service  standards  (or  performance  measures)  adopted  under 
Proposition  E wore  not  Intended  to  create  onerous  reporting 
requirements,  but  rather  to  provide  Muni  with  the  tools  needed  to 
create  a world-class  transit  service.  In  order  to  do  this  effectively, 
tlio  service  sundards  need  to  provide  Information  and  feedback 
that  Muni's  management  can  readily  use  to  help  shape  decisions  and 
policies  so  that  the  desired  outcomes  can  bo  achieved. 

While  Proposition  E specifically  stated  the  method  of  measurement 
and  goals  for  several  of  the  service  standards.  It  also  provided  some 
flexibility  witit  regard  to  the  way  In  which  other  standards  could  be 
measured  and  the  milestones  or  goals  to  bo  achieved.  When  not 
specified  by  Proposition  E,  the  MTA  Board  adopted  methods  and 
definitions  of  measurement  as  well  as  specific  goals  and  milestones 
for  each  of  the  service  standards.  Additionally.  Section  8A.I04  of 
the  City  Charter  allows  tlie  MTA  Board  to  vote  to  amend  any  of 
the  service  standards  (after  liolding  a public  hearing  on  any  such 
amendments). 

Miml's  Citizen's  Advisory  Committee  (CAC)  and  the  MTA  Board 
review  Muni's  performance  quarterly,  and  review  the  definitions  of 
mo.isiironiont,  methods  of  measuremont  and  tlic  goals  for  each  of 
tlio  service  standards  annually.  Muni  publishes  quarterly  Service 
Standards  Reports  which  include  a description  of  each  of  the  Service 
Standards  and  a summary  of  Muni’s  performance.  (These  reports 
are  .available  to  the  public  via  Muni's  web  site,)  These  reports  also 
nclude  .additional  performance  Information  that  Is  not  required 
by  Reposition  E.  but  Is  used  by  Muni  for  other  purposes,  such  as 
employee  incentive  pegrams, 
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Table  ES-I  summarizes  the  service  standards  that  are  required  by 
Proposition  E.  the  actual  measurements  that  are  made,  the  goals 
or  milestones  that  have  been  established,  and  overall  performance 
during  the  period  covered  by  this  review  (FY200 1 and  FY2002),  (For 
a more  detailed  discussion  of  performance,  please  see  the  body  of 
this  report.) 
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Table  ES- 1 : Proposition  E Service  Standards  and  Performance 
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Service  Standard  Prop  E Requirements 


A.  gysiipELiAttm?^  ‘ ~ 


Measurement 


Comments 


Goal  er  Milestone 


Systemwide  Performance 
(Audited  for  FY200I  and 
FY2002;  Unaudited  for  FY2003) 


la.  Sol  vehicles  that  run 
on  lime  according  to 
published  schedules 

On-time  performance 
slandard:65Sof 
vehicles  most  be  on-time 
Ino  more  than  1 minute 
early  or  four  minutes 
late  as  measured  against 
a published  schedule) 

Actual  arrival  times 
compared  against  published 
times  (quarterly  average 
for  systemwide,  mode,  and 
division) 

Specifically  measures 
what  is  required  by  Prop 
E. 

FY2001:  65S 
FY2002;  70S 
FY2003:  75S 
FY2004:  85S 

FY2001:  goal  achieved 
FY2002:  goal  achieved 
FY2003;  failed  to  achieve  goal 

La.  TV  UI  SCficOUIGO  S6rVIC0 

hours  that  are  delivered 
and  % ol  scheduled 
vehicles  that  begin 
service  at  the  scheduled 
lime 

Service  delivery 
standard:  96.5S  of 
scheduled  service  hours 
must  be  delivered,  and 
98.5S  of  scheduled 
vehicles  must  begin 
service  at  the  scheduled 
time 

- % operator  availability 
(quarterly  for  systemwide, 
mode,  and  division) 

- % equipment  availability 
(quarterly  (or  systemwide, 
mode,  and  division) 

-S  total  availability 
(quarterly  for  systemwide, 
mode,  and  division) 

-Sol  late  pull-outs 
(quarterly  lor  systemwide, 
mode,  mode  and  division) 

Specifically  measures 
what  is  required  by 
Prop  E;  however  Prop  £ 
does  not  state  that  % 
of  hours  delivered  for 
operator  availability  and 
equipment  availability  be 
reported  separately. 

FY2001;  96.5S 
FY2002:  97S 
FY2003;  97.5S 
FY2004:  98.5S 

FY2001:  failed  to  achieve  goal 
FY2002:  goal  achieved 
FY2003:  failed  to  achieve  goal 

sifl.  Missed  $efvic$  due 
to  either  insulficieni 
vehicles  or  driver 
unavailability  as 
a percentage  of 
scheduled  ser^ce  hours 

Missed  service  due 
to  either  insufficient 
vehicles  or  driver 
unavailability  as  a 
percentage  of  scheduled 
servtce  hours 

- % hrs  missed  due  to 
operator  availability 
quarterly  for  systemwide, 
mode,  and  division) 

- S hrs  missed  due  to 
equipment  availability 
Iquanerty  for  systemwide, 
mode,  and  division) 

- S of  total  hfs  missed 
(quarterly  for  systemwide, 
mode,  and  division) 

Specifically  measures 
what  IS  required  by 
’rop  E;  however  Prop 
Edoes  not  stale  that 
S of  hours  missed 
due  to  operator 
unavailability  and 
equipment  unavailability 
)e  reported  separately, 
'his  standard  is  simply 
an  inverse  ol  Service 
Standard  2a. 

FY2001:  3.5S 
FY2002:  3S 
FY2003;  2.5S 
FY2004:  I.5S 

FY2001;  failed  to  achieve  goal 
FY2002;  goal  achieved 
FY2003:  (ailed  to  achieve  goal 
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Systsmwide  Perfonnancs 


Vlill  IJi??iTni'l  Uri7lii  I IMI I ilyr”"'*  MPMU^manl  Comments  Coal  or  Milesliine  FY2fl02?u^Vu°^^ 


4n.%olvahiclos  Hint  pass 
piilillsliod  lima  polnli 
(luimu  monsmnmonl 
lioilods  unnblo  to  pith 
up  passoiiQois  tiuu 
to  crawilliip  willioiit 
liuliiy  followod  wlililn 
3inlnuioso(  lossliy 
imolhoi  vflhlclo  on 
tlio  snmofouln  will) 
apnco  lot  oil  wnilino 
linsjonoois 

S of  vohiclos  thni  pnss 
puhlishod  timo  points 
iluiinp  monsuiomonl 
poiiods  unoblo  to  pick 
up  possoniiors  duo 
lociowdiny  wlllioiil 
bniMD  loltuwod  williln 
3 nilnutos  or  loss  by 
nnollini  voliiclo  on 
llio  snmo  route  will) 
spaco  lor  nil  wnilino 
pnssonnois 

S of  vohiclos  possino 
hypossQiioors  duo  to 
ovorciowdod  voliiclo 
|i|uarloily  lor  syslomwido, 
modi),  mid  division) 

MTA  Board  adoplod  a 
sampling  niolhodology 
to  doiormino  porcent  of 
pass  ups  systemwide 
ond  for  oach  modo. 

MTA  Board  also  adoplod 
nspocllic  goal. 

uoBi  or  MiiMione 

FY2001:  goal  achieved 
FY2002;  goal  achieved 
FY2003:  failed  to  achieve  goal  for 
four  out  of  12  months 

Do.  mk  jiniloil  pnasonpoi 
lomi  fnclors 

Ponk  pniiodpn$9Qii[|or 
lond  Inciors 

Sof  pnssongois  onbonid 
voliiclo  ilini  oicood 
coiiibinod  sooting  ond 
standing  copncilylqumioily 
lor  syslomwido,  mode,  and 
division) 

MTA  Boord  adopted  a 
dollniiion  nnd  method 
of  moasuromoni  and  a 
spotilic  goal. 

£ 85%  of  combined 
seating  & standing 
capacity 

FY2001;  failed  to  achieve  goal 
FY2002;  failed  to  achieve  goal 
FY2003;  failed  to  achieve  goal 

On.  Mciufii  iioniiwnys 
niinlMst  .itlioilulixl 
Immlwnys  uii  nil  imllnl, 
axpiDM,  CIOSS'lOIVM, 
sncoiulmv  n«d  louilm 
linns  lui  iniiik,  linso, 
uvunlnii  mill  Inionlulii 
snivlciis 

Acliinl  hondwnys  ngnlnst 
scliudiiind  liofldwnys  on 
nil  iiidini,  oipioss,  cross 
town,  suconilnry  niid 
loodui  linos  lor  punk, 
bnsu.  uvoiiiiiu  mid  Into 
iilobl  survlcos 

Srovomio  vohiclos  nrilionng 
to  sclioduled  tionilwnys 
Iquntlotly  (or  syslomwido, 
modo.  nnd  division) 

MTA  Board  miopiod  n 
dolimlion  and  niolhod  of 
monsiiiomoiil  and  goal. 

FY200I:  i80% 
FY2002.  FY2003, 
FY3004;  a 85%  of 
the  lime  within  30%  or 
10  minutes  of  scheduled 
headway  (whichever  is 
lossi 

FY2001:  failed  to  achieve  goat 
FY2002;  failed  to  achieve  goal 
FY2003:  failed  to  achieve  goal 

71  VIIIMIHI  nvUMnUMliy 
mill  lolinliililyliiinmi 
tJisiniica  biilwiiuii 
(niliiio)  by  "uiilo 

S voliiclo  iivnilnbilily  nnd 
ruliobllity  (iiiumi  disiniico 
hulwami  Isiluio)  by 
nioilo 

S voliiclo  ovnilnbility 
(qumloily  lor  syslomwido. 
modo,  nnd  division) 

Only  ropoilSvaliicto 
nvnilobilily.  Roliobility  is 
topotlod  undot  Service 
Standard  Oa.  MTA 
Oomdndopiodo  specific 

goal. 

& 96.5%  vehicle 
eveilobilily 

FY2001;  goal  achieved  (except 
(or  light  rail) 

FY2002;  goal  achieved  (except 
for  light  rail) 

FY2003:  goal  achieved  (except 

for  tight  raill 

TnucES-J 


Proposition  E Municipal  Transportation  Quality  Review 


Pinal  Report 


Service  Standard 

Prop  E Requirements 

Measurement 

Comments 

Gosi  or  Milsstone 

Systeniwide  Piriormance 
(Audited  for  FY2001  end 
FY2DQ2:  Unaudited  for  FY2003) 

6a.  Unscheduled  absences 
by  operator,  mechanical 
and  administrative 
personnel 

Unscheduled  absences 
by  operator,  mechanical 
and  administrative 
personnel 

Sof  absences  not 
scheduled  in  advance  by 
personnel  (quarterly) 

MTA  Board  adopted  a 
definition  and  method 
of  measuremertt  and 
specific  goals.  Also 
use  data  for  employee 
incentive  goals. 

Annual  5S  reduction  of 
YTO  lor  Maintenance 
(Mechanical), 

10S  reduction  for 
Transportation 
(Operators),  and  5S 
reduction  for  Admin 

FY2001:  failed  to  achieve  goal  (or 
all  divisions 

FY2002:  Administration  Division 
failed  to  achieve  goal;  goal 
achieved  for  other  divisions 
FY2003.  goal  achieved  lor  all 
divisions 

9a.  Miles  between  roadcalls 
by  mode 

Miles  between  roadcalls 
by  mode 

Miles  beiweert  roadcalls 
or  mean  distance  between 
failures  (M08F)  (quarterly 
for  systemwide,  mode,  and 
division) 

MTA  Board  adopted  a 
definition  and  method 
of  measurement  and 
specific  goals. 

Increase  miles  between 
roadcalls  - specific  goals 
are  established  each 
fiscal  year  and  quarter 
(or  each  division 

FY2001  seven  out  of  ten 
divisions  achieved  goal 
FY2002.  five  out  ol  nine  divisions 
achieved  goal 

FY2003'  SIX  out  of  nine  divisions 
achieved  goal 

e.  SYSTEMi^^SiKKIvo 

V - • • 1 

1b.  Passengers  carried  by 
mode 

Passengers  earned  by 
mode 

- # of  people  boarding 
revenue  vehicles  (annually 
for  systemwide  and  model 

- H change  in  ridership  from 
previous  year  (annually  lor 
systemwide  and  model 

MTA  Board  adopted  a 
definition  and  method 
of  measurement  and 
specific  goafs. 

FY2001  and  FY2002: 
2^  increase  over 
previous  year 
FY2003  and  FY2004: 
224  M passengers 

FY2001;  goal  achieved 
FY2002:  goal  achieved 
FY2003'  failed  to  achieve  goal 

2b.  Fare  revenues 
generated  by  mode 

Fare  revenues  generated 
by  mode 

- Onboard  fare  collected  on 
revenue  vehicles  (annually) 

- Sales  of  individual  licitels 
(annually) 

- Sales  of  passes  (annually) 

- Muni's  share  of  regional 
passes  sold  (annually) 

MTA  Board  adopted  a 
definition  and  method 
of  measurement  and 
specific  goals. 

FY2001  and  FY2002: 
increase  fare  revenue  by 
$1.6M  over  previous  year 
FY2003:  $100  Min  fare 
revenue 

FY2004:  $117Minfare 
revenue 

FY200):  goal  achieved 
FY2002:  goal  achieved 
FY2003:  tailed  to  achieve  goal 
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Sarvlea  Standard  Prop  E Raqulrementa 


31).  Iloiits  mid  mllos 
opoinlinlliy  inodo 


41).  t)i[)Dn.i09  liicmiod  by 
moilu 


Homs  niid  milos 
ojiurnlodby  motlo 


Enpoosos  Intuiroilby 
inodo 


C.  STAFflWG  PEm  ORMANCr 


Meaiurament 


- #ol  vflliiclos'  lovonuo 
liouis  [onnuollybYiiioilo) 
-Hoi  vohiclos'  lovonuo 
milos  (oiumslly  by  model 


Only  (oiioiisvvholtioi  ot  not 
costs  wuiowilliin  budgol. 
Ooosiiol  report  ociunl 
ospoiisiis  iiicuifodby  moilo. 


Ic.  Mill  vnconclos  by 
poslllon  (vncniiclos 
iimininino  o»co 
promollons  nnil  now 
biros  linvo  boon 
dmiuclod  Iruni  lulliuos 
or  rusioimllons)  lot 
oncli  ilivisloii 


2c.  Alliiilon  tflius  lot  now 
iimploynos,  by  ilivlslon 
mill  luvol 


D^susroMEn  slih/ice^, 
(I.  Dovulopmoni  ol 
mimmlnimKiUinu  plan 
lilimtllyinp  spucitic 
piouimns  mul  piojocls 
Ihnlwill  pimmito 
IncronsciIpniimiiHni 


Nni  vnconclos  by 
position  (vncniiclos 
tomnininp  onco 
piomollons  nml  now 
bliDs  Imvu  boon 
dmliiclud  Irom  rnllroos 
or  rosliinniions)  lot  oncli 
tlivislon 


Alltilion  tnins  (or  now 
omplayuos,  by  division 
nml  IovdI 


Diidpiod  positions  (lor 
Iho  yooilnnd  potconi  ol 
vnconclos  (quntlotlyby 
division) 


<iol  nowbltosjolnnsos 
nml  volunioty  sopointions 
divided  by  lotntnumboi  ol 
novvhlius  in  soma  qumloi 


Commanti 


MTA  Board  Adopted  a 
dollnilion  and  moiliod 
ol  moosurement  and 
spocilic  pools. 


MTA  Board  adopted  a 
dolinitlon  and  method  ol 
monsmement  end  goal. 


Goal  or  Milestone 


fY200l  and  FY2002; 
1.2%  increaso  in  tevenuo 
miles  and  revenue  hours 
FV2003:  3.6M  hours  and 
25M  miles 

FY200<J:  3.5M  hours  and 
25M  milos 


Expenses  to  remain 
within  budgeled  amount 


SYStemwide  Performance 
(Audited  for  FY2001  and 
FY2002;  Unaudited  for  FY2003) 


FV2001:  goal  achieved 
FY2002;  goal  achieved 
FY2003;  goal  achieved  for  miles: 
tailed  to  achieve  goal  for  hours 


FY2001:  goal  achieved 
FY2002:  goal  achieved 
FY2003:  noi  slated  in  Service 
Standard  Report 


MTA  Board  adopted  o 
dclinilion  and  mothod 
ol  moosurement  and 
spocilic  goals. 


MTA  Board  odopiod  a 
dolimiion  and  moiliod 
ol  moasuiomont  and 
spocilic  goals. 


s 5S  vacancy  rale 


s lOS 


FY200I;  goal  achieved 
FY2002:  goal  achieved 
FY2003:  goal  achieved 


FY2001:  failed  to  achieve  goal 
FY2002:  foiled  to  achieve  goat 
FY2003:  lailed  to  achieve  goal 


Dovolopmont  o( 
niimifll  inmkolmg  pinn 
liloiiljlyiny  spocilic 
progrmns  and  piojocls 
that  will  promoto 
iticioflsod  pBtronngo 


Dovolop  and  publish 
Moikoling  PInn  miminlly 


MTA  Board  odopiod  a 
dolimtioii  ondmoihod  ol 
moasuiomeni  and  goal. 


Annual  Markeitng  Plan 
publishod 


FY2001:  goalachiavad 
FY2002;  goal  achieved 
FY2003.  not  Slated  in  Service 
Standard  Report 
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Service  Standard 

Prop  E Requirements 

Measurement 

Comments 

Goel  or  Milestone 

Systemwido  Psrformsnco 
(Audited  for  FY2001  end 
FY2002;  Unaudited  for  FY2003) 

2d.  Publication  and 
distribution  to  the 
public  of  schedules 
for  all  trips  taken 
by  all  vehicles 
that  shall  consist 
of  specific  arrival 
times  at  teimmals 
and  established 
intermediate  points 

Publication  and 
distribution  to  the  public 
of  schedules  lor  all  trips 
taken  by  all  vehicles  that 
shall  consist  of  specific 
arrival  times  at  terminals 
and  established 
intermediate  points 

Printing  and  distribution  of 
time  tables  lor  all  vehicle 
trips,  consisting  of  specific 
arrival  times  at  terminals 
and  intermediate  time  points 

MTA  Board  adopted  a 
defintlion  and  method  of 
measurement  and  goal. 

Printing  and  distribution 
of  time  table 

FY2001:  goal  achieved 
FY2002:  goal  achieved 
FY2003.  not  stated  in  Service 
Standard  Report 

3d.  Operator  conduct 
complaints 

Operator  conduct 
complaints 

- % of  Passenger  Service 
Reports  (PSRsI  resolved 
within  30  days 

- S change  in#  of 
Passenger  Service  Reports 
from  previous  yeartby  type 
of  complainll 

MTA  Board  adopted  a 
definition  and  method 
of  measurement  and 
specific  goals. 

Goal1;75S  ol  all 
Passenger  Service 
Reports  resolved  in  30 
days 

Goal  2:  lOH  reduction 
of  Passenger  Service 
Reports  annually  110^ 
reduction  from  previous 
year) 

FY2001:  goal  achieved  for 
complaint  resolution;  failed  to 
achieve  goal  for  # of  PSRs 
FY2002:  failed  to  achieve  goals 
FY2003:  goal  achieved  for 
complaint  resolution;  failed  to 
achieve  goal  lor  H of  PSRs 

4d.  Annual  passenger 
surveys  and  follow-up 
by  management 

Annual  passenger 
surveys  and  follow-up  by 
management 

- Passenger  Survey 
conducted 

- Employee  Survey 
conducted 

MTA  Board  adopted  a 
definition  and  method  of 
measurement  and  goal. 

Conduct  annual 
passenger  survey  and 
employee  survey 

FY20D1:  goal  achieved 
FY2002:  goal  achieved 
FY2003:  not  slated  in  Service 
Standard  Report 

5d.  Improvements  in  public 
information  regarding 
vehicle  delays  during 
operations  as  well 
as  any  general  user 
information  regarding 
system  moifilications, 
route  changes  and 
schedules 

Improvements  in  public 
information  regarding 
vehicle  delays  during 
operations  as  well 
as  any  general  user 
information  regarding 
system  modifications, 
route  changes  and  i 

schedules 

Was  not  developed  in 
FY2001  orFY2002  (in 
FY2004,  stated  that 
measurement  would 
’Assess  current  practices, 
develop  and  rmplement 
improvement  plan') 

Did  not  develop  specific 
measurement  until 
FY2004. 

Feedback  Irom  passenger 
surveys 

Rider  surveys  were  conducted, 
but  were  not  used  to  assess 
performance  under  this  measure. 
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SsrvIcD  Siandard 

Pro[i  E lUquIrinwnti 

Moaiuromsnt 

Commanli 

Goal  or  Milostone 

Systemwide  Performance 
(Audited  for  FY2001  and 
FY2002;  Unaudited  for  FY2003) 

Oil.  [lloMs  to  ImiKovo 
iltivni  itnitiinu, 
lucimicnl  na  wnll  ns 
nccliloni  lolloW'iifi 

Cllorls  lo  impfovu  dtivor 
Kolnlno,  loclinicol  ns 
wolf  ns  nccliloni  lolloW' 

U|1 

> /f  of  hours  ond  typo  ol 
Dtivor  Ttoinino  Iqunilerly 
omlfVTOI 

- # of  nccidonls,  numhoi  ol 
oporotof  pay  hours  wotkod, 
occidonl  tnlu  pur  100,000 
opotolor  puy  hours  worked 
Isysiumwido,  division  ntid 

yofli 

-tolnl  numbot  ol  occldcnls 
niso  roporiod  i)iiarioily| 

MTA  Board  adoplod  a 
dolinliion  and  method 
ol  rnoasurcinenl  and 
specific  oodIs. 

FY2001  and  FY2002: 

100.000  operator 
training  hours 
FY2003  and  FY2004: 

50.000  operator  training 
hours 

5S  accident  reduction 
over  previous  year  {all 
fiscal  yeaisl 

Goals  lor  operator  training  hours 
wers  achieved  in  all  three  fiscal 
years. 

Failed  lo  achieve  accident 
reduction  goals  for  all  three  years. 

7d.  Niimliofolciimn 
Inclduiilson  Miiiilcipnl 
riiillwny  volilcloa  0( 

In  Miinlcl|inl  Umlwny 
fncilillna 

Nmiibm  of  cdruu 
incldunis  on  Municipal 
Ifnllwny  volilclus  or 
In  Mi)iilcl|)nl  Rnilwnv 
Inclllllos 

Ho\  Inthlonl  loports  on 
Muni  voliiclos  or  piopoily 
(syslomwldo  by  quntlor  and 
mmunllyl 

MTA  Board  adopted  o 
dolimlioaand  method 
ol  moDsurcmoni  and 
specific  gools. 

5S  reduction  from 
pluvious  year 

Goal  was  achieved  in  all  three 
years. 

n-EMI‘LOV£CSATISFACTI 

ON 

1u.  NumliiK  nl  utlovniicoa 

Numliot  of  otlnvnncos 

H ol  omployoo  oiiovnnco 
toporis  lor  ttnnsit  opoioiors 
nnd  inointonanco  & misc, 
oniployoos  (quorloily  ond 
nnnunllyl 

MTA  Board  ndopled  n 
doliniiion  ond  method 
of  moosurcmonl,  but 
nospDcilic  goolwos 
odoplod. 

Quariorly  grievance 
ropoil 

No  spocilic  goal  was  adopted. 
Grievance  reports  were  published 
quarterly. 

2o.  Sjinoil  of  loaolutlon  of 
Oiinvnncoa 

Spaoii  ol  rosoliilinn  of 
Qilovnncos 

1 

I’oiconi  ol  Oporaiions  and 
Mnini.  & Misc.  Eiiiployoos 
OriovancQs  losolvud  wilhin 
30  dnys 

1 

MTA  Board  odoplod  o 
doliniiion  ond  molhod 
of  moflsuiomont  ond 
spocilic  goals. 

Resolve  75S  ol  internal 
grievances  within  30 
days 

For  all  quarters  in  which  data 
wero  reported,  the  goal  was 
achieved  lor  operator  grievances. 
Achievement  was  less  consistent 
lot  mainl.  and  misc.  employee 
grievances. 
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Service  Standard 

Prop  E Requirements 

Measurement 

Comments 

Goal  or  Milestone 

Systemwide  Performence 
(Audited  fer  FY2Q01  end 
FY2002;  Uneudited  fer  FY2003) 

3e.  iDngevily  of 
employnient 

Longevity  of  employment 

D of  employees  that  leave 
through  retirements, 
terminations  and  promotions 
(job  category,  annual) 

MTA  Board  adopted  a 
definition  and  method  of 
measurement,  however 
no  specific  goals  lor 
longevity  of  employment 
were  adopted. 

Annual  report  of  longevity 
of  employment 

Reports  were  published  in  FY2001 
and  FY2002, 

4e.  Employee  recognition 

Employee  recognition 

Tracking  of  all  employee 
award  programs 

MTA  Board  adopted  a 
delinition  and  method 
of  measurement  and 
specific  goals  lor  the 
number  of  awards  to  be 
made. 

Annual  achievement  of 
honorees  in  the  following 
award  programs  \H  of 
awards): 

(12)  Systemwide 
Operators  of  the  Month 
Award 

(4)  Transit  Supervisors  of 
the  Quarter  Award 
(4)  Finance  & Admm 
Employees  of  the  Quarter 
M2)  Maintenance 
Employee  of  the  Month 
(41  Maintenance 
Supervisors  of  the 
Quarter 

14)  Safety  & Training 
Employee  of  the  Quarter 
(4)  Accessibility  Employee 
of  the  Quarter 

Actual  number  of  awards  made 
were  not  included  in  the  Service 
Standards  Reports. 

Se.  Employee  education  and 
training  opportunities 

Employee  education  and 
training  opportunities  , 

1 

j!f  of  hours  of  training 
classes  by  type  of  training 

MTA  Board  adopted  a 
definition  and  method 
of  measurement  and  , 
specific  goals. 

50,000  annual  hours 
of  employee  training  or 
approximately  20  hours 
per  FTE 

FY2001:  goal  achieved 
FY2002:  goal  achieved 
FY2003;  failed  to  achieve  goal 
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An  Independent  Transportation  Quality  Review 

This  blonnIalTransportatlon  Quality  Review  mandated  by  Proposition 
E provides  yet  another  tool  that  Muni  can  use  to  continue  to  Improve 

Its  performance.  This  review  has  been  conducted  with  the  foliowint! 
goals  In  mind;  ® 

Help  Muni  assess  Its  progress  toward  the  goals  and  objec- 
tives of  Proposition  E and  develop  action  plans  for  enhancing 
performance  In  areas  where  there  may  be  some  deficiency. 

Evaluate  Munis  cstabllslicd  goals  and  performance  against  the 
letter  and  Intent  of  Proposition  E, 

Assess  whether  specific  Implementation  goals  and  methods 
and  dofin  tions  of  measurement  arc  appropriate  or  could  be 
Improved. 

• Provide  Independent  verincatlon  to  the  public  that  Muni  is 

co"du7e?  * •■''’•■'‘ys's  Pro- 

Thls  Transportation  Quality  Review  consists  of  four  main  elements; 

• Data  review  and  verification  of  performance  - Propo- 
sitlon  E requires  a routine  audit  of  Muni’s  quality  assunneo 
process  Including  an  audit  of  da«  collection  n;^ct~ 
Jorvico  standard  reporting.  This  Is  the  first  audit  of  Muni’s 
i-oportlng  on  service  standards  and  covers  fiscal  years  2001 

Munl^  Service  Standards  Reports  from  those  years  to  vcrlfV 

methods  of  moasuromom  specified  by  the  MTA  and  tint 
.^parted  service  standards  wc.-e  computed  co  v 
omatic  spot  chocks  of  original  sounro  data  and  oConn.ed 
racking  systems  and  prxjccdurcs  wore  used  to  determine  the 
accuracy  of  reported  service  standards  Durinn  th  l 
Of  OCO.0,.  „„d  NOVOO..O,. 
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responsible  for  data  collection  and  computation  to  review 
both  data  collection  and  computation  procedures  as  well  as 
the  actual  performance  data.  Appendix  A lists  the  Muni  staff 
who  were  Interviewed, 

Trends  analysis  - Auditors  reviewed  trends  in  data  and 
performance  achievement  over  the  two-year  audit  period, 
as  well  as  unaudited  data  and  performance  from  fiscal  years 
2003  and  2004,  Findings  from  this  trends  analysis  were  used 
to  develop  Action  Plans  for  those  areas  in  which  Muni's  per- 
formance could  be  improved. 

AuditorrecommendationsandAction  Plans -Auditor 

rcwmmendatlons  focus  on  ways  to  further  refine  or  improve 
performance  reporting  to  make  it  more  relevant  to  the  MTA. 
Muni  management  and  the  public.  Action  Plans  are  similar  in 
nature  to  auditors’  more  general  recommendations,  however 
■hoy  focus  more  spcdncally  on  ways  to  intprove  performance 
or  make  reporfing  more  accurate  and  relevant,  particularly 
or  those  service  standards  In  which  Muni  has  failed  to  meet 
hs  porfornaance  goals.  Although  the  recommendations  and 

77  =udit  period,  they  incor- 

porate any  changes  that  have  been  made  since  that  time.  Both 
■he  recommendations  and  Action  Plans  were  reviewed  with 

currl7h  H '’’“'"'u  '■““"’'"'"Nations  are  in  line  with 

current  budget  and  resource  constraints. 

Documenution  and  communication  of  results  -This 

r;ris~;r 
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Key  Findings  and  Recommendations 

Overall,  Muni  has  done  an  excellent  job  of  fulfilling  its  Proposition  E 
mandate.  Auditors  found  that  nearly  all  data  were  accurately  recorded 
and  reported,  and  that  reports  were  made  available  to  the  public  and 
reviewed  with  the  Citizens'  Advisory  Committee  and  MTA  Board 
quarterly.  Performance  was  generally  good  and  improved  over  time; 
although  there  are  some  areas  that  will  require  specific  attention  in 
order  to  achieve  established  performance  goals. 

The  auditors  also  identified  a number  of  opportunities  to  further 
streamline  and  improve  the  service  standards  in  order  to  make  them 
more  relevant  to  the  public,  the  MTA  Board  and  Muni  management. 
Because  data  do  not  always  yield  the  type  of  information  that 
decision  makers  can  use  to  develop  plans  and  actions  that  will 
enable  improvements  in  performance,  auditor  recommendations 
and  Action  Plans  focus  on  ways  that  Muni  can  amend  definitions  or 
methods  of  measurement  and  performance  goals  so  that  the  service 
standards  can  provide  information  that  will  serve  as  a truly  useful 
management  tool.  In  some  cases,  we  recommend  eliminating  service 
standards  that  fall  to  provide  truly  useful  information,  since  there 
is  little  value  in  collecting  data  if  they  cannot  be  transformed  into 
useful  information. 

Following  is  a summary  of  the  key  findings  and  recommendations  of 
this  report 

• Overall,  Muni  staff  who  provide  service  standard  data  to  the 
Superintendent  of  Performance  Monitoring  and  Analysis  need 
to  provide  updated  data  whenever  they  become  available.  In 
a few  Instances,  data  obtained  from  Muni  staff  did  not  match 
exactly  data  that  were  reported  in  the  Service  Standards 
Reports.  This  was  primarily  due  to  the  feet  that  data  were 
subsequently  updated,  but  the  Service  Standards  Reports  were 
not  updated  as  well. 
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System  Reliability 

• To  improve  on-time  performance  and  meet  the  goals  es- 
tablished under  Service  Standard  la.  it  will  be  necessary  to 
reduce  the  incidence  of  early/sharp  operations  (more  than 
one  minute  early).  It  may  also  be  necessary  to  make  schedule 
and  route  revisions  on  those  lines  that  have  chronically  poor 
performance. 

• For  Service  Standards  la  and  6a,  it  may  be  advantageous  to 
reallocate  resources  used  to  perform  terminal  checks  so  that 
a greater  number  of  line  checks  can  be  made  (once  early  de- 
partures from  terminals  are  no  longer  a significant  problem). 
The  goal  of  such  a reallocation  would  be  to  increase  the 
frequency  of  checks  in  order  to  provide  additional  seasonal/ 
quarterly  observations  on  each  mode.  This  does  not  suggest 
that  terminal  leaving  times  are  unimportant  to  reliability,  but 
rather  that  counts  could  be  more  effectively  taken  elsewhere 
on  the  line. 

• For  Service  Standards  la  and  6a.  it  may  be  reasonable  to 
consider  measuring  on-t/me  performance  (schedule  adherence) 
only  for  those  services  with  headway  intervals  of  more  than 
10  minutes,  and  measure  heodwoy  Adherence  (time  between 
transit  vehicles)  on  those  services  with  headway  intervals  of 
10  minutes  or  less.  Because  these  changes  would  require  an 
amendment  to  the  City  Charter,  it  may  not  be  practical  to 
implement  them  in  the  near  term. 

• In  order  to  achieve  established  goals  for  the  percentage  of 
scheduled  service  hours  that  are  delivered  in  Service  Standard 
2a.  it  will  be  necessary  to  address  the  loss  of  service  due  to 
operator  unavailability.  It  would  be  prudent  for  Operauons 
Managers  to  draft  a plan  to  consider  reallocation  of  operator 
resources  between  modes  (where  appropriate)  in  order  to 
equalize  any  missed  service  systemwide.  It  is  probable  that 
specific  shortages  in  operator  availability  in  any  single  mode 
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will  not  bo  addressed  In  a systemwide  target  but  should  be 
minimized  on  a mode  and  division  basis,  Notwltlisundlng 
labor  agreements  and  past  practices,  Operations  Managers 
should  take  all  necessary  steps  to  maintain  the  highest  levels 
of  on-street  service  provision  when  there  are  spikes  in  opera- 
tor availability,  ^ 

Service  Standard  3a  should  bo  eliminated  since  It  Is  simply  the 
Inverse  of  Service  Standard  2a  and  Is  completely  duplicative. 

Service  Standard  -la  should  be  coordinated  with  Service 
Standard  5a  so  that  all  lines  Identified  as  exceeding  85%  of 
capacity  during  peak  periods  are  monitored  as  part  of  Service 
Standard  4a,  Additional  data  available  through  die  Passenger 
Service  Reports  database  should  bo  utilized  to  conduct  follow- 
up trafnc  checks  on  lines  Identified  as  having  chronic  ntes  of 
pass-ups  or  peak  period  loads  that  exceed  85%  of  capacity. 

• For  Service  Standard  6a.  the  method  of  measuring  headway 
adherence  should  bo  modified  so  that  It  Is  more  relevant  to 
high  frequency  service  (headway  Intervals  of  10  minutes  or 

B,  Syjiem  Performantc 

’ should b. 

mmt  oo  llm  cm  servo  os  .1  surrogoto  for  ridorship  during 

periods  of  (.,1V,  srablllty.  Combined  wills  oilier  indlcitors  It 
is  possible  .0  cnicubie  nver^e  fere  per  pessenger, 
nie^emen.  tool  when  looking  e,  elierneiL  fe, e siruerrl" 
Idle  this  Indicator  should  continue  to  be  used,  the  current 
practice  of  changing  the  bonchm.uk  for  faro  rovonuo  annually 
based  on  the  prior  year's  actual  experience  does  not  allow  J 
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reasonable  comparison  across  time.  Rather  than  continuing 
to  alter  the  benchmark,  It  Is  recommended  that  the  milestone 
be  restatcd;‘'Fare  revenue  should  increase  by  1.5%  per  year 
compared  with  prior  year  performance."  Changing  the  mile- 
stone In  this  way  will  allow  for  comparisons  over  time  and 
will  serve  as  a better  Indicator  of  the  need  to  adjust  fares  and 
fare  structure, 

• Milos  and  hours  of  service  (Service  Standard  3b)  are  a direct 
reflection  on  the  available  budget.  It  is  not  reasonable  to 
expect  that  in  trmes  of  budget  constraints  miles  and  hours 
of  service  will  continue  to  Increase,  or  even  remain  constant 
Miles  and  hours  of  service  are  important  inputs  to  the  Na- 
tIonalTrans  t Database  (NTD)  reporting.and  will  be  calculated 
annually  whether  or  not  they  continue  to  be  used  as  a per- 
formance measure.  Consequently,  the  CAC  and  MTA  Board 
should  consider  eliminating  this  service  standard. 

The  goal  and  Information  presented  for  Service  Standard  4b 
have  limited  value  to  the  public  or  to  Muni  Itself.  In  fact,  Mun, 
bus  no  opportunity  to  oxcced  its  budget  without  receiving 

vlZ  dlTl  “ = Civ  =8=ncy  cennot  open 

mZ  u «PWS«  to  budget  by 

th°,  Mun  f " It  recommended 

thn.  Mu  I truck  ully  nllocsted  cosb  per  hour  of  service  by 

To  come  m "““''I  ^un 

Stalling  Performance 
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sure  that  maintenance  and  operator  positions  arc  filled.  Con- 
sequently. the  auditors  recommend  that  this  service  standard 
be  limited  to  Transit  Operators  and  Maintenance  personnel, 
or  that  the  goals  for  other  divisions  (exclusive  ofTransii  Op- 
erators and  Maintenance  personnel)  be  revised  so  that  they 
can  be  reasonably  met  given  budgetary  constraints. 

• In  order  to  track  the  attrition  rate  for  new  hires  in  Service 
Standard  2c,  attrition  rates  should  either  be  reported  at  the 
end  of  new  employees'  probationary  periods  or  within  one 
year  after  hire.  To  calculate  attrition,  only  those  employees 
who  separate  after  six  months  or  one  year  of  service  should 
be  counted.  Furthermore,  there  is  no  information  provided  in 
this  measure  about  why  employees  are  voluntarily  separating 
from  Muni  service.  If  it  is  possible  to  obtain  this  information 
from  the  Position  Control  System  database,  then  it  should  also 
be  summarized  and  included  in  this  measurement. 

D.  Customer  Service 

• For  Service  Standard  I d,  marketing  priorities  should  be  made 
within  budget  constraints  and  with  the  goal  of  increasing 
patronage;  data  from  ridership  surveys  should  be  used  to 
evaluate  the  success  of  marketing  efforts. 

• For  Service  Standard  2d.  the  current  process  of  schedule 
preparation  is  cumbersome,  time  consuming,  and  relies  on 
the  background  and  knowledge  of  one  key  staff  person.  The 
capabilities  of  the  new  Trapeze  scheduling  software  should  be 
fully  utilized  in  order  to  simplify  and  streamline  the  process 
of  schedule  preparation. 

• For  Service  Standard  3d,  operator  conduct  complaints  need 
to  be  more  clearly  measured  and  reported,  realistic  goals  for 
reducing  the  number  of  operator  conduct  complaints  should 
be  established,  and  the  efficiency  and  accuracy  of  complaint 
information  uptake  should  be  improved. 
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• To  be  truly  useful.  Service  Standard  5d  should  be  divided  Into 
two  goals  to  distinguish  between  routine  information  provided 
in  real  time,  and  longer-term  information  provided  as  part  of 
service  or  schedule  changes  and/or  construction  projects. 

• In  order  to  meet  the  accident  reduction  goals  established  in 
Service  Standard  6d,  it  will  be  necessary  to  develop  a compre- 
hensive approach  and  coordinated  effort  to  reduce  accidents 
across  divisions  and  departments.  Driver  training  was  found 
to  be  sufficient  Therefore,  the  focus  of  accident  reduction 
efforts  should  be  enforcement  of  safety  rules  and  procedures 
and  accident  prevention  through  thorough  review  of  all  ac- 
cidents. Additionally,  it  would  be  useful  to  report  types  of 
accidents  according  to  those  that  occur  during  revenue  and 
non-revenue  hours,  and  those  that  were  determined  prevent- 
able and  non-preventable. 

Employee  Satisfaction 

• In  addition  to  the  Employee  Satisfaction  service  standards, 
Muni  is  also  measuring  employee  satis^ction  via  its  annual 
Employee  Survey.  The  results  of  this  survey  should  be  incor- 
porated into  this  group  of  service  standards  since,  in  many 
respects,  it  provides  a better  measure  of  employee  satisfaction 
than  the  current  group  of  service  standards. 

• For  Service  Standard  I e,  a goal  or  benchmark  for  the  number 
and  type  of  grievances  should  be  established  so  that  manage- 
ment can  more  effectively  monitor  changes  in  grievance  rates. 
Additionally,  reporting  on  the  types  of  grievances  filed  (rather 
than  just  the  total  number)  would  provide  valuable  informa- 
tion, 

• For  Service  Standard  2e,  the  method  for  reporting  speed  of 
resolution  of  grievances  for  non-operator  grievances  needs  to 
be  closely  examined  and  monitored  to  ensure  that  information 
is  reported  correctly  in  the  future. 
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Service  Standard  3o  provides  almost  no  Information  about 
omployoo  satisfaction.  If  anything,  It  Is  more  a measure  of 
staffing  performance,  since  long-term  employees  (and  relatively 
minimal  stiff  turnover)  reduce  the  need  to  hire  and  train 
now  employees.  The  auditor  therefore  recommends  that  this 
measure  bo  eliminated  from  the  Service  Standards  Report  or, 
If  retained,  moved  to  the  Staffing  Performance  section,  Alter- 
natively, a more  useful  measure  of  employee  satisfaction  may 
be  the  average  length  of  service  upon  voluntary  employee 
separation. 

For  Service  Standard  do,  the  actual  number  and  typo  of  annual 
and  quarterly  awards  made  should  bo  included  In  the  Service 
Standards  Report. 

For  Service  Standard  So,  all  employee  training  hours  should 
bo  tracked  In  a centralized  system  In  order  to  facilitate  fu- 
turo  reports  and  audits.  Additionally.  Muni  should  consider 
mo.isuring  and  reporting  on  the  perceived  effectiveness  and 
usefulness  of  training  through  employee  surveys  and  tninine 
evaluation  forms,  ® 
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A.  System  Reliability 

This  group  of  service  standards  measures  system  reliability  - the 
‘extent  to  which  transit  vehicles  arrive,  depart  and  are  able  to 
^accommodate  all  passengers  at  expected  times  and  frequencies. 
ISystem  reliability  is  the  most  significant  factor  In  attracting  choice 
Riders,  as  riders  are  unlikely  to  depend  on  a system  that  is  unreliable. 
Poor  system  reliability  can  significantly  lengthen  the  amount  of  time 
(passengers  spend  both  waiting  for  a transit  vehicle  and  traveling  to 
llheir  final  destination.  It  can  also  result  in  crush  loads  and  pass  ups. 
|v/here  transit  vehicles  are  at  capacity  and  unable  to  accommodate 
^iting  passengers.  This  situation  often  leads  to  further  delays  and 
breakdowns  in  service  reliability.' 

Koposition  E specified  the  following  service  standards  in  order 
to  improve  and  maintain  the  reliability  of  Muni  transit  service. 
Collectively,  these  performance  measures  give  on-going  snapshots 
and  the  ability  to  diagnose  service  delivery  problems. 

• la.  % of  vehicles  that  run  on  time  according  to  published 
schedules  (no  more  than  4 minutes  late  or  I minute  early) 
measured  at  terminals  and  established  intermediate  points. 

• 2a.  % of  scheduled  service  hours  that  are  delivered  and 
percentage  of  scheduled  vehicles  that  begin  service  at  the 
scheduled  time. 

• 3a.  Missed  service  due  to  either  insufficient  vehicles  or  driver 
unavailability  as  a percentage  of  scheduled  service  hours.  (This 
is  simply  an  inverse  of  the  preceding  service  standard.) 

• 4a.  % of  vehicles  that  pass  published  time  points  during 
measurement  periods  unable  to  pick  up  passengers  due  to 
crowding  without  being  followed  within  3 minutes  or  less  by 
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another  vehicle  on  the  same  route  with  space  for  all  waiting 
passengers. 

• 5a.  Peak  period  passenger  load  factors. 

• 6a.Actual  headways  against  scheduled  headways  on  all  radial, 
express, cross-town. secondary, and  feeder  lines  for  peak. base, 
evening,  and  late  night  services. 

• 7a.%  vehicle  availability  and  reliability  (mean  distance  between 
failure)  by  mode. 

• 8a.  Unscheduled  absences  by  operator,  mechanical  and  admin- 
istrative personnel. 

• 9a.  Miles  between  roadcalls  by  mode. 
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Key  Recommendations 

Tho  following  arc  key  recommendations  based  on  the  findings  of  our 
audit  of  FY200I  and  FY2002  Service  Standards  data  and  on  trends 
observed  In  unaudited  FY2003  and  FY2004  Service  Standards  data; 

• To  Improve  on-tIme  performance  and  meet  the  goals  es- 
tablished under  Service  Standard  la.  it  will  be  necessary  to 
reduce  the  Incidence  of  carly/sharp  operations  (more  than 
one  minute  early).  It  may  also  be  necessary  to  make  schedule 
and  route  revisions  on  those  lines  that  have  chronically  poor 
performance. 

• For  Service  Standards  la  and  6a,  It  may  bo  advantageous  to 
reallocate  resources  used  to  perform  terminal  checks  so  that 
a greater  number  of  line  checks  can  be  made  (once  early  de- 
partures from  terminals  arc  no  longer  a significant  problem), 
Tho  goal  of  such  a reallocation  would  bo  to  Increase  the 
frequency  of  checks  to  provide  additional  scasonal/quarterly 
observations  on  each  mode.  This  does  not  suggest  that  termi- 
nal dcpariitro  times  arc  unimportant  to  reliability,  but  rather 
that  counts  could  bo  more  effectively  taken  elsewhere  on  the 
lino. 

• For  Service  Standards  I a and  6a.  It  may  bo  reasonable  to  con- 
sider measuring  on-time  performance  (schedule  adherence) 
only  for  those  services  with  headway  Intervals  of  more  than 
10  minutes,  and  measure  headway  adherence  (time  between 
transit  vehicles)  on  those  services  with  headway  Intervals  of 
10  nilnutos  or  less.  Because  these  changes  would  require  an 
amendment  to  tho  City  Charter,  It  may  not  bo  pncclcal  to 
Implement  them  In  tho  near  term. 

• In  order  to  achieve  established  goals  for  tho  percentage  of 
scheduled  service  hours  that  are  delivered  in  Service  Standard 
2a,  It  will  bo  necessary  to  address  tho  loss  of  service  due  to 


operator  unavailability.  It  would  be  prudent  for  Operations 
Managers  to  draft  a plan  to  consider  reallocation  of  operator 
resources  between  modes  (where  appropriate)  in  order  to 
equalize  any  missed  service  systemwide,  It  Is  probable  that 
specific  shortages  In  operator  availability  in  any  single  mode 
will  not  be  addressed  in  a systemwide  target  but  should  be 
minimized  on  a mode  and  division  basis.  Notwithstanding 
labor  agreements  and  past  practices.  Operations  Managers 
should  take  all  necessary  steps  to  maintain  the  highest  levels 
of  on-street  service  provision  when  there  are  spikes  in  opera- 
tor availability. 

• Service  Standard  3a  should  be  eliminated  since  it  is  simply  the 
Inverse  of  Service  Standard  2a  and  is  completely  duplicative 

• Service  Standard  4a  should  be  coordinated  with  Service 
Standard  5a  so  that  all  lines  identified  as  exceeding  85%  of 
capacity  during  peak  periods  are  monitored  as  part  of  Service 
Standard  4a.  Additional  data  available  through  the  Passenger 
Service  Reports  database  should  be  utilized  to  conduct  fol- 
low-up traffic  checks  on  lines  chronically  identified  as  having 
pass-ups  or  peak  period  loads  that  exceed  85%  of  capacity. 

• For  Service  Standard  6a,  the  method  of  measuring  headway 
adherence  should  be  modified  so  that  it  is  more  relevant  to 
high  frequency  service  (headway  intervals  of  10  minutes  or 
less). 
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la.  % of  vehicles  that  run  on  time  according  to  published  schedules  (no  more  than  4 minutes 
late  or  I minute  early)  measured  at  terminals  and  established  intermediate  points. 


jGoat:  .1  ' 

[No:less;than;85%.or( July  1/2004  * /•  >=;- 

Purpose: 

To  measure  schedule  adherence 

Definition  of  :i 
Measurement: 

1- 

, 

■ v;'j 

1 - ■ ■ 1 
1 

1 ' ■ " ■ i 

■ 1 

1FY2001:  Monthll^rheasurement  of  the  5 most  patronized  routes  for  each  division.  Such  checks 
;shajl  be  conducied  no  less  often  than  10  weekdays  and  weekends  per  month  on  5 different  routes 
,pe'r-month-in  each'division,  beginning  with  the  most  patronized  route,  progressing  in  the  order  of 
^declining  route  patronage,  and.ending  with  the  least  patronized  route.  The  cycle  shall  then  be 
Jrepeated.;’  '';^'  ; 

'FY2002:  Each  line'Will  be  checked  twice  a year.  Such  checks  shall  be  conducted  no  less  often 
jthan  10  weekda^St’ano'weekends  per  check.  An  annual  checking  schedule  shall  be  established  for 
•the;foutes.*The  order  in  which  the  routes  are  checked  will  be  determined  monthly  through  a random 
!selectidn''prdcess.«‘To  the  extent-automated  systems  can  be  substituted  at  less  cost  for  such  • 
^checks,  or  the'measurement  of  any  performance  standard,  such  systems  must'be  used. 

• - > 

Method  of 

Check  the  designated  lines  using  criteria  of  -1  minute  to  -f-d  minutes.  "Periods  of  time  includes 

Measurement: 

morning  rush  |6  a.m.  to  9 a.m.)  midday  (9  a.m.  • 4 p.m.)  evening  rush  (4  p.m.  to  7 p.m.)  and  night 
|7p.m.  to  1 a.m.).  Supervisory  personnel  shall  conduct  a one-hour,  on-time,  and  load  standard  check 
at  a maximum  load  point  at  mid-route  during  all  four  time  periods  stated  above.’ 

Milestones: 

;i:Y2tlOT  - FY2002 

70%/-- 

From  Muni  Stfyce  Sta/xiords  RefiOfl  Ff200l  and  FY2002. 
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Reported  Performance 

Tabic  A- 1 summarizes  the  quarterly  results  by  mode  for  fiscal  years 
2001,  2002,  2003  and  the  first  and  second  quarters  of  fiscal  year 
2004.  The  Jump  In  performance  for  the  light  rail  and  cable  car 
modes  between  the  third  and  fourth  quarters  of  FY200I  was  due 
to  a change  In  measurement  procedure.  Beginning  In  the  fourth 
quarter  of  FY200I,  scheduled  arrival  times  were  no  longer  linked  to 
scheduled  run  numbers  for  cable  car  and  light  rail  modes.  Instead, 
checks  wore  made  to  determine  whether  any  vehicle,  rather  than 
a specific  run  number,  arrived  at  scheduled  times.  (This  change 
was  approved  by  the  MTA  Board.)  The  original  FY2002  goal  was 
lowered  from  75%  to  70%.  Consequently,  the  established  goal  was 
met  for  both  FY200I  and  FY2002  for  systemwide,  fourth  quarter 
performance. 

Subsequent  to  the  audit  period,  on-time  performance  held  steady 
In  FY2003  (unaudited  figures)  but  failed  to  meet  the  75%  on-tIme 
standard  and  appears  to  have  declined  In  the  first  quarter  of  FY 
2004. 
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Table  A- 1 On-Tirr>e  Performance,  FY200 1 - FY2004 


1st 

Quarter 

2nd 

Quarter 

3rd 

Quarter 

4th 

Quarter 

FY  I 
Goa! 

1 FY2001  On-Time  Performance  1 

Motor  Coach 

58.6% 

60.3% 

68.3% 

67.8% 

1 

65% 

Trolley  - - 

i..-65.1% 

r;6i.3%; 

66a% 

light  Rail 

42.9% 

45.5% 

42.5% 

63.0% 

CablOiCsr . *■' 

- 18iO%T 

B7.2%- 

Systemwide 

50.2% 

53.4% 

50.6% 

65.5% 

1 FY2002  On-Time  Performance  I 

Motor  Coach 

65.1% 

69.4% , 

68.6% 

69.9% 

70% 

Trolley'-  • 

76.1%: 

^■.71:3% 

76.1%. 

Light  Rail 

59.8% 

65.1%  1 

67.7% 

67.8% 

CablB^Car 

67.0%  1 

m • 

' r 70;9% 

Systemwide 

66.9% 

70.9%  1 

69.5%  1 

71.9% 

1 FY2003  On-Time  Performance  | 

Motor  Coach 

70.4% 

68.8% 

67.2% 

71.3%  \ 

75% 

Trolley 

73.}%: 

74:6%. 

. 74.6% 

73.5% 

Light  Rail 

64.3% 

66.6% 

66.9% 

69.1% 

Cable' Car 

' 72.6% 

^£8.2%. 

.67.0% 

Systemwide 

70.1% 

70.7% 

70.0% 

70.9% 

FY2004  On-Time  Performance  I 

Motor  Coach 

1 69.4% 

65.9% 

85% 

1 

1 

Trolley 

,\68.8%. 

..  JIM 

light  Rail 

62.5% 

678% 

Cable  Car 

86.0% 

1 68J% 

Systemwide 

67.3% 

' 68.0% 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

mi 

The  following  changes  in  sampling  methodology  were  approved  by 
the  MTA  Board  and  subsequently  implemented  in  FY200I: 

• Light  rail  {LRV)  and  cable  car  divisions  were  combined  into 
one  rail  sampling  rotation  with  results  reported  separately; 
eight  checks  per  month  were  reduced  to  five  or  four  checks 
per  month. 

• Three  motorcoach  divisions  were  combined  tnto  one  motor- 
coach  sampling  rotation;  1 5 checks  per  month  were  reduced 
to  five  checks  per  month. 

• Two  trolleycoach  divisions  were  combined  into  one  trolley- 

I coach  sampling  rotation;  ten  checks  per  month  were  reduced 
to  five  checks  per  month. 

Lines  were  sampled  within  each  mode  according  to  ranked 
patronage  as  described  in  the  definition  of  measurement.  At  least 
ten  weekday  checks  were  performed  during  each  month  for  motor 
coach  and  trolley  coach  modes.  Light  rail  and  cable  car  modes  were 
in  compliance  with  the  definition  of  measurement  for  the  months 
they  were  sampled.  All  time  periods  were  sampled  in  accordance 
with  the  definition  and  method  of  measurement. 

Ff2002 

The  selection  of  routes  was  coordinated  wi^  the  Data  Collection 
Plan  (DCP),  adopted  by  Muni  to  satisfy  the  annual  NTD  (and 
(previously  UMT,^FTA  Section  15)  ridership  reporting.  The 
PflOOl  methodology  for  sampling  15  iines/checks  per  month  was 
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modifed  for  FY2002  to  include  nine  motorcoach  lines,  three  to  four 
trolleycoach  lines,  and  two  to  three  rail  lines.  A random  selection 
process  replaced  the  ranked  ridership  rotation  process.  Specific 
locations  and  time  periods  conformed  to  the  DCP  and  included,  as 
a minimum,  the  Prop  E defined  time  periods. 

V'erifrcat/on  of  Reported  Performance  During  the 
Audit  Period 

Based  on  an  audit  of  the  fourth  quarter  FY2001  and  FY2002 
submittals  by  the  Muni  Analysis  Coordinator,  performance  appears 
to  have  been  reported  correctly  during  the  audit  period. 
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Problems  or  Issues  Identified  During  the  Audit  Period 

While  there  were  no  specific  problems  or  Issues  with  reported  performance  or  the  method  or  definition  of  measurement,  a more  detailed 
analysis  of  performance  for  specific  lines  Identified  several  Important  Issues.  An  analysis  of  the  schedule  adherence  reporting  over  FY200I 
and  FY2002  shows  some  continuing  trends  in  those  Individual  routes/lines  with  the  poorest  results.  The  following  table  highlights  cumulative 
findings  for  the  poorest  performing  lines  by  mode: 


Tabic  A-2  Poorest  Performing  Lines  for  On-Tlme  Performance,  FY2001  - FY2002 


Motor  Coieh 
LInei 

2001 

<60%  on- 
time 

2001 

<66%  on-tlme 
>10%  airly 

2002 

<60%  ee-llme 

2002 

<70%on-tlBM 
> lOSoorly 

:'9nx  1 

1 1 

X 

1 » 

' * 

1G 

X 

I17  I 

L-  X:^ 

X I 

1 

10 

X 

X 

?0 

X ; 

. . r 

t --X' 

20 

X 

"jp  w.  ■ , 

' X 

/ • w 

30 

X 

X 

i3(lAX 

r-  ■ 

&X'^-A-. 

51 

X 

X 

X 

X 

;1  X 

^ X 

SOX 

X 

SIX 

X 

It.  X(  ' ' 

Ml^x 

B2X 

X 

ISO  I 

1 'X»*'  1 

X 

Trolley  Coach 
Lines 


2001 

<50%  on. 
lime 


2001 

<65%  onlime 
>10%  early 


2002 

<S0%ontimB 


2002  . 
<70%ontMM 
> I0%esrly 


14 


21 


49  . . . • .c- 


Light  Rail 
Lines 


2001 

<50%  on- 
lime 


2001 

< 65%  on-time 
>10%  early 


2002 

<50%  on-time 


2002 

< 70%Dn-tim 
> I0%early 


Cable  Car 
Lines 


2001 

<50%  on- 
time 


59 


2001 

<65%  on-time 
>10%  early 


2002 

< 50%  on-time 


60 


61 


2002 

< 70%on-tme 
>10%early 
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The  systemwide  schedule  adherence  standard  cannot  reasonably  be 
attained  without  specific  correaive  action  to  the  poorest-performing 
lines.  Given  current  trends,  it  is  improbable  that  the  attainment  of 
85%  schedule  adherence  will  be  reached  in  FY2004.  When  examined 
further,  the  auditor  found  that  failure  to  meet  on-time  performance 
was  due  in  large  part  to  early  arrivals.*  This  is  encouraging,  since 
many  delays  in  service  result  from  street  conditions  that  are  beyond 
Muni's  control.  Early  departures  result  from  many  causes,  and  may 
be  cultural,  behavioral,  or  intentional  - but  most  of  the  causes  are 
entirely  within  Muni's  control.  If  early/sharp  operation  is  dramatically 
curtailed,  then  achievement  of  85%  on-time  performance  could  be 
possible  for  all  modes. 

In  FY200I,  for  the  81  lines  for  which  on-time  performance  was 
reported,  all  had  a minimum  of  3%  early/sharp  observations,  with  an 
average  of  1 7. 1 % early/sharp  for  motor  coach.  1 4.9%  early/sharp  for 
trolley  coach.  36.4%  early/sharp  for  light  rail,  and  32.2%  early/sharp 
for  cable  car.  All  of  the  early  arrivals  cited  above  were  at  least  two 
minutes  or  more  ahead  of  schedule. 

In  FY2002.  for  the  80  lines  for  which  on-time  performance  was 
reported,  all  but  one  had  a minimum  of  3%  early/sharp  observations, 
With  an  average  of  18.9%  early/sharp  for  motor  coach.  14.4%  early/ 
sharp  for  trolley  coach,  23.0%  early/sharp  for  light  rail,  and  17.2% 
early/sharp  for  cable  car.  While  systemwide  on-time  performance 
improved  by  14.5%,  the  improvement  in  early/sharp  arrivals  was  less 
dramatic  - only  6.6%.  Additionally,  it  appears  that  for  the  rubber 
tire  combined  motor  coach  and  trolley  coach  modes,  incidence  of 
early/sharp  operation  actually  inaeosed  from  15.6%  to  16.7%.  It 
should  be  noted  that  improvement  in  the  rail  modes  in  both  on- 
bme  achievement  and  reduction  of  early/sharp  observations  was 
due  primarily  to  the  change  in  measurement  method  cited  earlier. 


‘tirt,  njiWm  li  » ilpxifitant  contributor  to  fiilurei  to  meet  on-oms  poffofTTunte  goili  for  other 
mnui  ijencies  u wdL  Ibid.,  p 3-46. 
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Muni  staff  have  recognized  that  early/sharp  arrivals  are  a significant 
impediment  to  achieving  on-time  performance  goals  and  have  taken 
the  following  steps  in  order  to  address  this  and  other  on-time 
performance  problems: 

• Street  Inspectors  wilt  be  deployed  at  terminals  (to  the  extent 
possible)  to  insist  on  schedule  adherence  and  prevent  early 
departures.  In  order  to  make  the  best  use  of  the  limited  num- 
ber of  Street  Inspectors,  Inspectors  will  now  be  responsible 
for  checking  multiple  modes  at  key  transfer  points. 

• Responsibility  for  on-time  performance  will  be  more  heavily 
focused  at  the  division  level.  Division  Managers  will  have  fewer 
administrative  duties  and  will  be  able  to  concentrate  more 
on  directly  overseeing  Street  Supervisors. 

• Schedules  will  be  reviewed  to  make  sure  they  properly  coin- 
cide with  running  times. 

• Next  Bus  technology  will  be  used  to  monitor  the  on-time 
performance  of  certain  lines  in  real  time  so  that  Street  Inspec- 
tors can  make  necessary  adjustments  before  more  significant 
on-time  problems  arise. 

• Muni  will  continue  to  work  closely  with  the  Department  of 
Parking  and  Transportation  to  clear  busways  more  quickly. 

• Daily  discussions  and  reviews  of  safety  and  on-time  perfor- 
mance with  operators.  Central  Control. and  Street  lnspe«ors 
will  continue.  Muni  staff  will  also  continue  to  meet  with  union 
representatives  in  order  to  obtain  their  support  for  meeting 
on-time  performance  goals. 

• An  education  and  incentive  campaign  is  planned  that  will  In- 
form transit  operators  about  on-time  performance  goals  and 
actual  achievement,  and  will  reward  those  operators  who  have 
best  on-time  performance. 
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Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  the  MTA  adjusted  the  Definition 
of  Measurement  for  FY2004  by  changing  the  first  sentence  to  read, 
"Each  line  will  be  checked  at  least  once  In  each  six-month  period." 

Action  Plan 

Given  the  continued  need  for  on-timo  performance  Improvement, 
the  recommendations  In  this  Action  Plan  focus  on  early  arrivals  and 
on  corrective  actions  for  the  poorest  performing  routes.  The  trends 
In  data  observed  for  Service  Standards  la  and  6a  (actual  headways 
against  scheduled  headways)  concurrently  require  both  on-street 
attention  to  supervision  and  preparation  of  new  schedules.  The 
modal  matrix  under  "Problems  or  Issues  Idcntlfled"  Identifies  lines 
with  the  highest  priority  for  both  schedule  evaluation/  remediation 
and  supervision. 

Incidence  of  Early/^harp  Obicrvaiions 

Attention  should  be  directed  to  reducing  the  Incidence  of  early/ 
sharp  operation  and  thereby  Increasing  dramatically  the  on-time 
performance,  Findings  for  both  audit  years  are  significant; 
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Given  the  Proposition  E measurement  definition  for  schedule 
adherence  that  allows  a window  of  +I/-4  for  attaining  "on-time" 
status,  those  percentages  of  early/sharp  cited  above  exceeded  the 
+ 1 allowance  by  a significant  amount.  All  such  observations  were  at 
least  two  minutes  or  more  ahead  of  schedule.  To  evaluate  schedule 
adequacy  (I.e.,  to  measure  the  degree  of  appropriate  running  time), 
recovery  time,  and  headway  (capacity),  ahead  of  schedule  (early/ 
sharp)  operation  must  be  curtailed.  Significant  on-street  supervisory 
resources  must  be  deployed  to  monitor  and  manage  both  end 
terminal  departures  and  mid-point  timepoint  schedule  adherence.  It 
is  apparent  that  if  early/sharp  operation  is  curtailed,  then  achievement 
of  85%  on-time  performance  is  within  target  in  all  modes. 
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leduk  Evaluation/ftemediatiort 

uni  is  fortunate  to  have  an  In-house  Data  Colleaion  Plan  that  incorporates  NTD  Passenger  Boarding/Ridership 
mpling  with  Proposition  E system  reliability  performance  sampling  Muni’s  commitment  to  adhere  to  the  Data 
Dllection  Plan  enables  periodic  updating  of  boarding  data  for  each  route/line  as  well  as  timepoint-level  running 
ne  data.  To  that  end.  new  baseline  data  are  collected  each  quarter  for  several  lines  across  various  modes.  The 
ditor  identified  26  lines  that  incurred  less  than  65%  on-time  adherence  and  greater  than  1 0%  early/sharp  arrivals. 
ie  extent  to  which  chose  lines  have  received  baseline  updates  and  analysis  is  as  follows: 
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Analysis  of  baseline  data  by  the  Analysis  Coordinator  within  the 
Schedules  Deparement  has  been  an  on*golng  function.  Those  lines 
Identified  with  deficient  running  itmc/recovery  lime  have  been 
quantified.  The  required  vehicle  hours  and  peak-to-base  vehicles  to 
Implement  a new  schedule  and  remedy  deficiencies  are  as  follows; 
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Annualized  platform  (operator)  costs  for  those  target  lines  listed 
above  would  be  approximately  $378,000.  Variable  costs  for 
maintenance  and  possibly  capiol  costs  for  added  peak  vehicles  have 
not  been  estimated. 

Route  Evaluation 

Route  length  (one-way  mileage)  and  enroute  turns/congestion  points 
arc  a source  of  on-goIng  schedule  conflict.  Evaluation  of  each  low- 
performing  route  for  schedule  deficiencies  should  likewise  include 
route  segment  analysis  to  determine  how  significant  congestion 
points  contribute  negatively  to  schedule  adherence.  If  any  congestion 
point  contributes  more  than  10%  one-way  running  time  variability, 
then  It  Is  assumed  that  rccovcT7  will  be  Insufficient  to  cover 
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this  condition  and  enroute  schedule  adherence  will  exceed  the  on- 
time  window.  Several  options  are  available  for  remediation; 

• Add  a "control  point"  mid-route  where  significant  conges- 
tion has  occurred  prior.  The  control  point  should  preferably 
be  at  a major  iransfer/generator  location. 

• Consider  segmenting  the  line  where  prevailing  demands  on 
both  ends  are  not  matched  and  the  costs  of  adding  recovery 
lime  for  two  new  termini  is  offset  by  the  headway  changes/ 
cycle  time  reduction  on  one  segment. 

• Consider  route  deviations  when  congestion  occurs  at  spe- 
cific locations  during  specific  time  periods. 

• Consider  preferential  right-of-way/transit  lane  designation 
where  specific  congestion  occurs  during  specific  time  periods 
(All  transit  preferential  signal  treatments,  including  signal  pre- 
emption, should  be  Identified). 

Targeted  Improvetnenii 

Implementing  a comprehensive  and  on-going  schedule  revision 
program  Is  the  mission  of  all  schedule  departments:  however, 
operational  needs  and  resource  constraints  often  preclude  this 
regularized  approach  to  maintaining  effeaive  operating  schedules. 
Although  the  specific  lines  identified  previously  as  under-performing 
may  be  prioritized  for  both  supervision  and  new  schedules,  they 
Individually  may  not  add  on-time  improvement  sufficient  to  raise  both 
the  modal  and  systemwide  percentage  significantly.  For  example,  the 
motorcoach  89  line  attaining  90%  on-time  status  versus  the  current 
(FY2002)  80%  would  not  contribute  as  much  as  the  motorcoach  47 
attainment  of  a similar  10%  improvement  (FY2002  on-time  at  55%), 
However,  collectively  the  15  under-performing  motorcoach  lines,  if 
improved  up  to  85%,  would  contribute  to  a 6%  Improvement  in  the 
mode  from  68.4%  to  74.5%. 
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Prioritizing  light  rail  (LRV)  for  reduction  of  early/sharps  by  I S%  (from 
23  % to  8%)  would  move  1 .6 1 2 observed  trips  to  on-time  attainment, 
and  significantly  raise  the  LRV  mode  to  80.6%.  Similarly,  prioritizing 
cable  car  to  meet  scheduled  headways  (observed  trips  and  not  runs 
currently  are  reconciled  to  scheduled  times)  throughout  the  day 
would  significantly  raise  on-time  attainment  from  69.1%  to  at  least 
85%.  Note  that  the  on-time  window  of  + I/-4  applies  to  the  day  base 
scheduled  headway  of  8 minutes. 

Sampling  Resources  - 

Reallocation  of  Terminal  Checking  (o  Increased  NTD  Checking 

Once  early  departures  from  terminals  is  no  longer  a significant 
problem,  it  may  be  appropriate  to  eliminate  terminal  checks  so  that 
a greater  number  of  line  checks  can  be  performed.  Although  the 
Proposition  E measurement  requirement  included  checks  at  terminals 
and  established  timepoints,  the  checking  resources  expended  to 
collea  from  terminals  is  about  twice  that  required  for  established/ 
Intermediate  timepoints.  The  Proposition  E mid-point  sampling 
plans  are  coordinated  with  Muni’s  NTD  sampling  plans  to  minimize 
the  checking  resources  expended.  The  Data  Collettion  Plan  for  FY 
2002/2003  (August  1, 2002)  indicates  that  9.720  hours  per  year  of 
checking  is  required  for  the  Proposition  E program  at  4.S  locations 
per  line.  Additionally.  3,432  hours  per  year  of  checking  is  required 
for  the  NTD  program  at  two  locations  per  line.  Clearly,  a significant 
74%  of  the  combined  programs  is  allocated  to  Proposition  E. 

The  resulting  difference  in  schedule  adherence  (between  terminals 
and  intermediate  timepoints)  is  only  about  1%.  however.  Thus,  while 
checking  terminals  provides  a validation  of  the  checks  at  intermediate 
timepoints  and  identifies  trips  that  leave  the  terminal  off-schedule, 
actual  results  indicate  that  such  a validation  may  not  be  necessary. 
It  would  appear  prudent  to  increase  the  sampling  of  intermediate 
timepoint  checks  in  lieu  of  the  terminal  checks  with  the  goal  of 
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sampling  each  line  each  quarter  rather  than  the  current  twice  yearly 
target. 

The  auditor  recommends  that  the  portion  of  Proposition  E checking 
hours  allocated  to  terminal  checks  (roughly  4.320  hours  per  year) 
be  reallocated  to  mid-point  locations.  The  goal  of  such  a reallocation 
would  be  to  increase  the  frequency  of  checks  to  provide  additional 
seasonal/quarterly  observations  on  each  mode.  Poorly  performing 
routes  should  be  targeted  for  additional  observations  as  well.  If  the 
approximately  4,320  hours  currently  allocated  to  terminal  checks 
were  reallocated  to  the  current  NTD  program  of  3.432  hours.  It 
would  more  than  double  the  frequency  of  checks  made  under  the 
NTD  program. 

Streamlining  the  Service  Standards 

On-time  performance  is  typically  used  to  measure  the  reliability 
of  transit  services  operating  at  headv/ays  that  arc  greater  than  10 
minutes.  For  services  operating  at  headways  of  10  minutes  or  less, 
headway  adherence  is  typically  used  to  measure  service  reliability.’ 
From  a passenger’s  perspective,  schedule  adherence  matters  when 
headways  are  longer  than  10  minutes,  since  passengers  rely  on 
scheduled  arrivals  and  departures  when  planning  their  trips.  For 
headways  that  are  10  minutes  or  less,  passengers  no  longer  rely  on 
schedules,  since  they  know  a vehicle  will  arrive  shortly  after  they 
arrive  at  their  stop.  Service  reliability  might  be  more  accurately 
measured  by  not  measuring  on-tIme  performance  for  high  frequency 
(headway  intervals  of  10  minutes  or  less)  transit  services,  but  by 
measuring  headway  adherence  instead  (see  the  “Action  Plan" 
section  under  Service  Standard  6a).  Because  these  changes  would 
require  an  amendment  to  the  City  Charter,  it  may  not  be  practical 
to  implement  them  in  the  near  term. 


’IWd.^3-47 
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Coordination  with  Other  Goals  and  Objectives 

On-timc  performance  Is  closely  related  to  the  other  performance 
goals  within  this  group  of  service  standards.  The  steps  taken  to 
Improve  on*tlmo  performance  should  be  closely  coordinated  with 
actions  taken  to  achieve  other  performance  measures,  particularly 
those  related  to  headway  adherence  (6a),  delivery  of  scheduled 
service  (2a)  and  safety  (6d). 
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2a.  % of  scheduled  service  hours  that  are  delivered  and  percentoge  of  scheduled  vehicles  that 

begin  service  at  the  scheduled  time. 


Goal; 

.No  less  than  98.5%  on  July  1, 2004 

Purpose; 

To  measure  service  hours  through  available  operators  and  available  equipment,  along  with 
the  percentage  of  equipment  available  for  service. 

Definition  of 
Measurement: 

Monthjy.measurement  of  the  percent  of  total  available  hours  for  service  measuring  7 . 
bperators.and.equipment  and  percentage  of  equipment  available  daily. 

Method  of 
Measurement: 

80th  operators  and  equipment  are  measured  as  to  the  total  number  of  hours  in  senrice 
as  a percentage  of  the  total  scheduled  hours.  Data  come  from  the  online  Dispatching 
System.  Measurement  of  the  vehicles  that  begin  service  at  the  scheduled  time  will  be 
provided  from  the  8 a.m.  and  6 p.m.  'Not-Out  Report'  generated  by  Central  Control  and 
will  show  the  percent  of  vehicles  that  went  out  at  the  scheduled  time  lor  both  the  a.m. 
and  p.m.  pullout. 
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Reported  Performance 

At  the  end  of  FY200I.  Muni  was  unable  to  meet  Its  goal  of  96.5% 
delivery  of  scheduled  service  hours,  either  systemwide  or  for  any 
particular  mode  or  division,  except  for  the  Kirkland  motorcoach 
division.  The  loss  of  hours  was  due  primarily  to  unavailability  of 
operators,  rather  th.in  equipment  There  was  significant  improvement 
In  FY2002,  however,  and  the  goal  of  97%  delivery  of  scheduled 


service  hours  was  met  for  most  divisions  (except  cable  car)  and 
systemwide. 

In  FY2003,  the  systemwide  goal  of  97.5%  was  not  met  although 
performance  appeared  to  improve  according  to  data  from  the  first 
quarter  of  FY2004.  Table  A-3  summarizes  the  fourth  quarter  results 
for  fiscal  years  2001, 2002  and  2003  and  the  second  quarter  results 
for  FY2004. 


Table  AO  Delivery  of  Scheduled  Service  Hours,  FY200 1 - FY2004 
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DeUvarod 
due  to 
Equipment 
Unavailable'- 


99.97% 


99.92% 


99.99% 


99.97% 


99.88% 


99.85% 


99.91% 


99.73% 


99.70% 


99.90H 


3S  of  Total 
Scheduled  \ 
Hours 
Delivered 


94.54% 


93.34% 


93.55% 


HHrs 
DeUvarod 
due  to 
Operator  ] 
Unavailable 


95.83% 


95.62% 


36.77% 


97.48% 


97.87% 


98.99% 


94.31% 


73.32% 


94.52% 


95.00% 


97.02% 


98.41% 


97.69% 


99.28% 


98.85% 


96.22% 


97.18% 


" %Hrs 
. Delivered 
due  to 
Equipment 
Unavailable 


99.95% 


93.88% 


99.99% 


99.97% 


99.97% 


99.93% 


99.94% 


99.90% 


100% 


■%  of  Totally 
SeheduHi. 

Hours 
■ Deliversii 


95.78‘, 


95.48r. 


95.00% 


96.9S-: 


98.38^^ 


97.66% 


9922-: 


9B.55:- 


96.22‘> 


99.95% 


37.13% 
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A goal  for  the  percentage  of  late  pull-outs  (or  percentage  of 
scheduled  vehicles  that  begin  service  at  the  scheduled  time)  was  not 
established.  As  shown  in  the  following  table,  the  percentage  of  late 
pull-outs  did  not  exceed  0.6S%  for  any  division  or  systemwide  in 
Richer  R'200 1 , Pl'2002  or  FT2003  (unaudited  results). 

Table  A-3  Percentage  of  Late  Pull-Outs  (of  all  scheduled 


runs  that  went  into  service),  FY200I  - FY2003 


lFY2Q03fAvGragC- 

Motor  Coach 

I Flynn 

0.06% 

0.01% 

0.07% 

Woods 

0.15% 

0.22% 

0.55% 

I Kirkland 

0.65% 

0.25% 

0.27% 

Trolley 

I Potrero 

0.10% 

0.06% 

0.16% 

1 Presidio 

0.00% 

0.07% 

0.04% 

LRV 

0.41% 

0.20% 

0.12% 

Cable  Car 

0.43% 

0.00% 

0.28% 

Systemwide 

0.23% 

0.14% 

0.24% 

Muni  Sovice  Swnd«<di  Re^r\  Quarjer  FY200I.  fY2002  end  Ff2003 
Diu  in  IbIIcs  hive  net  been  ludited. 


Ven7?cot/on  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

During  the  audit  period,  data  were  entered  on  a daily  basis  into 
the  Automated  Dispatch  system  by  the  division  dispatchers.  The 
dispatchers  followed  regularized  procedures  for  showing  runs 
Without  a driver  assignment  and/or  without  equipment  available. 
Division  Managers  were  responsible  for  reviewing  automated 
dispatch  data. 
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The  Automated  Dispatch  software  provided  a daily  aggregation 
of  data  on  operator  and  vehicle  availability.  This  information  was 
tabulated  to  provide  monthly  and  quarterly  assessments.  The 
percentage  availability  was  calculated  using  data  from  the  Automated 
Dispatch  software  and  scheduled  service  information  from  the 
Schedule  Department 

Verification  of  Reported  Performance  During  the 
Audit  Period 

The  position  responsible  for  performing  this  function  was  eliminated 
in  the  current  FY2003-2004  budget  the  function  Is  currently 
performed  by  a manager  in  the  scheduling  section  of  the  Operations 
Division.  Procedural  documentation  of  Proposition  £ reporting  for 
FY200I  and  FY2002  was  not  available.  Audit  verification  of  initial 
source  data  was  accomplished  by  query  of  the  Auto  Dispatch  system 
and  aggregation/summarization  was  accomplished  by  manual  means 
utilizing  hard  copies  of  weekly  missed  service  reports.  Based  on  this 
information,  the  percent  availability  of  operators  and  equipment  for 
Pr'2002  appears  to  have  been  reported  correctly. 

Problems  or  Issues  Identified  During  the  Audit  Period 

Other  chan  difficulties  encountered  due  to  staff  changes,  no  other 
problems  or  issues  were  identified.  Although  performance  goals 
were  achieved  in  FY2002,  future  performance  achievement  appears 
to  be  problematic. 
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Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  adjusted  the  milestones  for 
scheduled  service  hours.  The  FY2003  and  FY2004  milestones 
were; 

• FY2003  -97,5% 

• FY2004-98.S% 

Act/on  Plan 

Develop  a Plan  lor  Haxtmliing  Service  Provision 
During  low  Availabiliiy  Periods" 

The  milestone  targets  adopted  by  MTA  for  FY200I  and  FY2002  for 
this  service  standard  were  consistent  with  achieving  the  FY2004 
attainment  of  No  less  than  98.5%  of  service  hours  delivered  on 
July  1. 2004.’’  The  progress  between  FY200I  of  94.53%  systemwide 
service  hours  delivered  to  FY2002  of  97.80%  of  systemwide  service 
hours  delivered  was  significant.  However,  unaudited  fourth  quarter 
results  from  FY2003  Indicate  thatprogress  is  not contlnulng.slnceonly 
94,52%  of  scheduled  service  hours  were  delivered  systemwide. 

It  Is  realistic  to  assume  that  in  the  future  MTA  will  experience  various 
"spikes"  In  operator  availability  due  to  changes  In  the  rates  of  Claiming 
Industrial  and  Claiming  Industrial  for  Assault  (i.c..  operator  absences 
duo  to  workers  compensation)  and  fluctuations  In  the  hiring  process 
duo  to  potential  cuts  In  support  services,  It  would  be  prudent 
that  Operations  Managers  draft  a plan  to  consider  reallocation  of 
operator  resources  between  modes  (where  appropriate)  In  order  to 
equalize  any  missed  service  systemwide.  It  Is  probable  that  specific 
shortages  In  operator  availability  In  any  mode  will  not  be  addressed 
In  a systemwide  target  but  nonetheless  should  be  minimized  on  a 
mode  and  division  basis.  Notwithstanding  labor  agreements  and 
past  practices.  Operation  Managers  should  take  all  necessary  steps 
to  maintain  the  highest  levels  of  on-street  service  provision  when 
there  are  spikes  in  operator  availability. 
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3a.  Missed  service  due  to  either  insufficient  vehicles  or  driver  unavailability  as  a 
percentage  of  scheduled  service  hours. 


Goal: . . ' 

•No  less  than  V.5%  on-July'-l,  2004 

Purpose: 

To  measure  service  hours  through  available  operators  and  available  equipmertt. 

Definition'of  • 
Measurement: 

jMonthlymeasurement  of  the.percent  of  total  available  hours  for  service  measuring  operators  and 
lequipment'and  perceritage  of  equipment  available  daily. 

Method  of 
Measurement: 

Both  operators  and  equipment  are  measured  as  to  the  total  number  of  hours  in  service  missed  as  a 
percentage  of  the  total  scheduled  hours.  Data  comes  from  the  online  Dispatching  System. 

Milestones:  s-,; 

;FY2001,  ?FY2002-.- 

scheduled  service  hoursideti^^^^  • ' . 

from  Muni  Stnla  Standards  Report,  FY200 1 andfy2002. 
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Reported  Performance 

Tills  service  standard  Is  the  Inverse  of  Service  Standard  2a.  Rather 
than  the  percent  of  scheduled  service  hours  that  are  delivered,  this 
service  standard  reports  the  percent  of  scheduled  service  hours  that 
arc  missed  as  .a  result  of  Insufficient  vehicles  or  driver  unavailability. 

At  the  end  of  FY200I.  Muni  was  unable  to  meet  Its  goal  of  no  more 
than  3.5%  of  scheduled  service  hours  missed,  either  systemwide  or 
for  any  particular  mode  or  division,  except  for  the  Kirkland  motor 
conch  division.  Missed  hours  were  due  primarily  to  unavailability  of 


operators,  rather  than  equipment  There  was  significant  improvement 
in  FY2002,  however,  and  the  goal  of  no  more  than  3%  of  scheduled 
service  hours  missed  was  met  for  most  divisions  (except  cable  can 
and  systemwide. 

Any  improvement  gains  appear  to  have  been  lost  in  FY2003.  since 
the  goal  was  not  met  by  any  of  the  divisions  or  systemwide.  Second 
quarter  results  for  FY2004  do  not  indicate  much  improvement 
Table  A-5  summarizes  the  fourth  quarter  for  fiscal  years  2001, 2002 
and  2003  and  the  second  quarter  results  for  FY2004. 


Table  A-5  Percentage  of  Missed  Service,  FY200 1 - FY2004 
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1.75% 


7,93% 
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41h  Quarter  FY2003 


FY2003  Goal:  2.6% 


% llrs 
Missed 
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009% 
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1.51%. 


1.85% 
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2nd  Quarter  FY2D04 


FY2004  Coal:  1.6% 
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yerification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

During  the  audit  period,  data  were  entered  on  a daily  basis  into 
the  Automated  Dispatch  system  by  the  division  dispatchers.  The 
tispatchers  followed  regularized  procedures  for  showrng  runs 
vithout  a driver  assignment  and/or  without  equipment  available, 
division  Managers  were  responsible  for  reviewing  automated 
iispatch  data. 

The-  Automated  Dispatch  software  provided  a daily  aggregation 
of  data  on  operator  and  vehicle  availability.  This  information  was 
tabulated  to  provide  monthly  and  quarterly  assessments.  The 
percentage  availability  was  calculated  using  data  from  the  Automated 
Bispatch  software  and  scheduled  service  information  from  the 
Schedule  Department 

Verificat/on  of  Reported  Performance 
During  the  Audit  Period 

The  responsible  position  for  performing  this  function  v/as  eliminated 
In  the  current  Pf2003-2004  budget;  the  function  is  currently 
performed  by  a manager  in  the  scheduling  section  of  the  Operations 
Pivision.  Procedural  documentation  for  Proposition  E reporting  for 
FY200I  and  Pf2002  was  not  available.  Audit  verification  of  initial 
source  data  was  accomplished  by  query  of  the  Auto  Dispatch  system 
and  aggregation/summarization  was  accomplished  by  manual  means 
^iizing  hard  copies  of  weekly  missed  service  reports.  Based  on  this 
Information,  the  percent  availability  of  operators  and  equipment  for 
FY2002  appears  to  have  been  reported  correctly. 
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Prob/ems  or  Issues  Identif/ed  During  the  Audit  Period 

Other  than  difficulties  encountered  due  to  staff  changes,  no  other 
problems  or  issues  were  identified.  Although  performance  goals 
were  achieved  in  FY2002.  future  performance  achievement  appears 
to  be  problematic. 

Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  adjusted  the  milestones  for 
missed  scheduled  service  hours.  The  FY2003  and  FY2004  milestones 
were: 

• FY2003-2.5% 

• FY2004-1.5% 

Auditor  Recommendations 

This  service  standard  should  be  eliminated  since  it  is  simply  the 
inverse  of  Service  Standard  2a.  The  Citizen’s  Advisory  Committee 
(CAC)  has  also  recommended  that  this  Service  Sondard  be  removed, 
since  it  is  a duplicate  of  Standard  2a. 
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4a.  % of  vehicles  that  pass  published  time  points  during  measurement  periods  unable  to  pick 
up  passengers  due  to  crowding  without  being  followed  within  3 minutes  or  less  by  another 
vehicle  on  the  same  route  with  space  for  all  waiting  passengers. 


Gual: 

Less  than  5% 

Purpose; 

To  measure  crowding  in  vehicles. 

Dotiiiition  of 
Moiisuromont; 

^ i 

rV2001:  Monthly  measurement  of  the  5 most'pattonized  routes  f6r^e'ach'divisi6n.-''Such  chicks  shall ‘be'condiicfetf'’ 
no  less  often  than  10  weekdays  per  month  on  5 differentroutes  per-mpnfhTn  each 'division,  beginning  with  the^ 
most  paljjpiied  rouje.^^gressing  in  the  order.of  decliningToutepatronage,  and  endin^y/rtfi'the'lej^aironjzed'  ' 
route,  the.cycle  shall  then  be  repeated.Fy2002;  Each  line  will  be.checked  twice  a;ye3r.vSuch:checks^all*^'  '“1 
^becondji^^yess  Q^ti^  and  weekends  per.clieck?*  AVannuaT^checking  schedule  shatl.be^'  Xi 

oslabiishBdj.wtn  th^Iries  to  be  checked  each  mon^andomly  selecteli.  To  the  exteiit  automated  systems  can  be  • 
substituted  at  loss  cost  for  such  checks,  or  the  measurerpent  of  any  perfo[niance'Standard,isuch  systems  rnust'be  ’ 
used.  . 

Method  ol 
Monsiitomonl; 

Check  the  designated  lines  using  criteria  of  -1  minute  to  +4  minutes.  "Periods  of  time  includes  morning  rush  (6 
a.m.  to  9 a.m.)  midday  (9  a.m.  • 4 p.m.)  evening  tush  (4  p.m.  to  7 p,m.)  and  night  |7p.m.  to  1 a.m.}.  Supervisory 
personnel  shall  conduct  a one-hour,  on-time,  end  load  standard  check  at  a...  FY2001:  .maximum  load  point  at 
mid-route  during  ell  four  time  periods  stated  above.FY2002;  ...maximum  load  point  at  mid-route  during  all  four 
time  periods  stated  above.  Pass-ups  shall  be  checked  during  such  times  at  one  stop  before  the  maximum  load 
point 

Milestones; 

FY2001'  FY20D2  , , -•  -.V  . " ^ 

l8ssilmn5%'  Lessthan5?o  ..  "W-  j 

S(arxAirdiKrf>Mxry200l  «vfry2002. 
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Reported  Performance 

In  PT200 1 .fewer  than  0.35%  of  vehicles  sampled  passed  up  passengers 
due  CO  overcrowding  without  being  followed  by  another  vehicle  able 
to  accommodate  all  passengers  within  three  minutes.  In  the  fourth 
quarter  of  Ff 200 1 , the  pass  up  percentage  was  zero.  In  Pf2002,  the 
highest  pass-up  percentage  was  0.7 1 % and  the  lowest  was  0%.  Thus, 
in  both  FY200 1 and  FY2002,  the  goal  of  lower  than  a 5%  pass-up  rate 
was  not  only  achieved  but  surpassed.  The  highest  for  any  of  the  lines 

Table  A-6  Percent  of  Pass  Ups  (from  checks  at  MLP  for 
the  five  most  patronized  lines),  FY200 1 - FY2004 


From  Mvm  Seiwe  Standanli  Qu«ner  fY200l.  mOOl  FT200J  and  and  2nd  Quantr  mOCU 

Oao  cn  iiatia  hive  not  b««n  audited. 
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sampled  was  5,56%  for  the  9AX  San  Bruno  (November  2000)  and 
5.88%  for  the  I AX  California  A Express  (July  200 1 ). 

The  percentage  of  pass-ups  spiked  during  several  months  in  FY2003, 
and  were  relatively  high  for  the  M-Oceanview  and  the  N-Judah 
during  the  first  and  second  quarters  of  FY2004  Table  A-6  presents 
systemwide  pass-up  results  for  fiscal  years  2001.  2002.  and  2003 
and  for  the  lines  measured  during  the  first  and  second  quarters  of 
FY2004. 
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Verification  of  Definition  and  Method  of  Measurement  During  the  Audit  Period 


moo  I 

The  following  changes  In  sampling  methodology  were  approved  by 
the  MTA  Board  and  subsequently  Implemented  in  FY200I: 

• Light  rail  (LRV)  and  cable  car  divisions  were  combined  into 
one  rail  sampling  rotation  with  results  reported  separately; 
eight  checks  per  month  were  reduced  to  five  or  four  checks 
per  month. 

• Three  motorcoach  divisions  were  combined  into  one  motor- 
coach  sampling  rotation;  IS  checks  per  month  were  reduced 
to  five  checks  per  month. 

■ Two  trollcycoach  divisions  were  combined  into  one  trolley- 
coach  sampling  rotation;  10  checks  per  month  were  reduced 
to  five  checks  per  month. 

Lines  were  sampled  within  each  mode  according  to  ranked 
patronage  as  described  In  the  definition  of  measurement.  At  least 
ten  weekday  checks  were  performed  during  each  month  for  motor 
coach  and  trolley  coach  modes.  Light  rail  and  cable  car  modes  were 
In  compliance  with  the  definition  of  measurement  for  the  months 


they  were  sampled.  All  time  periods  were  sampled  in  accordance 
with  the  definition  and  method  of  measurement. 

fy2002 

The  selection  of  routes  was  coordinated  with  the  Data  Collection 
Plan  (OCP),  adopted  by  Muni  for  provision  of  data  for  internal  planning 
purposes  and  to  satisfy  the  annual  NTD  (and  previously  UMTAl 
FTA  Section  IS)  ridership  reporting.  The  FY200I  methodology  for 
sampling  15  lines/checks  per  month  was  modified  for  FY2002  to 
Include  nine  motorcoach  lines,  three  to  four  trolleycoach  lines,  and 
two  to  three  rail  lines.  A random  selection  process  replaced  the 
ranked  ridership  rotation  process.  Specific  time  periods  conformed 
to  the  DCP  and  included,  as  a minimum,  the  time  periods  defined  in 
Proposition  E. 
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Verification  of  Reported  Performance 
During  the/iudit  Period 

An  audit  of  Pf200l  and  FY2002  data  confirmed  that  pass-up  data 
was  reported  correctly  for  all  but  the  fourth  quarter  of  FY200I 
For  the  fourth  quarter  of  FY200I,  zero  pass  ups  were  reported  In 
the  Service  Standards  Report,  however  source  data  showed  that 
five  pass  ups  occurred  during  that  quarter.  It  is  likely  that  updates 
or  changes  to  data  were  not  transmitted  to  the  Superintendent  of 
Performance  Monitoring  and  Analysis. 

For  the  second,  third  and  fourth  quarters  of  FY200I,  the  number 
of  checks  at  the  maximum  load  point  reported  in  the  4th  Quarter 
FY200I  Service  Standards  Report  were  incorrect.  Following  are  the 
Correa  values: 


fiJTd'iiuaner^ 

s3rdrQusrteri 

v4th  QuarteT 

Number  of  Checks  at  MLP 

8,881 

7,373 

9.633 

These  reporting  problems  were  considered  minor  and  correcable. 


Prob/ems  or  /ssues  identified  During  the  Audit  Period 
No  additional  issues  or  problems  were  identified. 

Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  adjusted  the  definition  of 
measurement  for  pass-ups,  as  follows; 

* Pass-ups  shall  be  conducted  no  less  often  than  10 
weekdays  per  month.  At  the  beginning  of  each  quarter, 
supervisory  personnel  will  review  all  the  lines  checked  in  the 
previous  quarter,  and  identify  the  five  lines  with  the  highest 
load  faaors,  and  the  time  period  these  load  factors  occurred. 
Supervisory  personnel  will  then  check  those  five  lines,  during 
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the  time  period  that  the  high  load  faaor  occurred,  each  month 
during  the  coming  quaaer  for  pass-ups.  Supervisory  person- 
nel will  check  to  see  if  any  PSRs  for  pass-ups  were  made  for 
any  of  those  five  lines,  and  if  the  location  of  the  pass-up  was 
recorded.  If  it  was  recorded,  supervisory  personnel  will  use 
that  point  as  the  point  to  check  for  pass-ups.  If  there  are  no 
locations  recorded,  supervisory  personnel  will  use  the  stop 
just  before  the  MLR 

Auditor  Recommendations 

Performance  results  have  been  reported  in  two  different  formats. 
The  4th  Quaaer  FY200I  Service  Standards  Report  contained 
a system-wide  table  summarized  by  quarter  which  included  the 
following  information:  number  of  pass-ups,  percentage  of  pass-ups, 
number  of  checks  at  MLR  and  the  lines  with  more  than  two  pass-ups. 
The  4th  Quaaer  FY2002  Service  Standards  Report  did  not  contain 
a cumulative  (each  quaaer  and  annual)  systemwide  table.  Because 
the  sampling  plan  for  lines/modes  varies  by  month,  both  a quarterly 
and  an  annual  performance  summary  should  be  presented. 

Two  changes  are  recommended  in  repoaing: 

• Include  a systemwide  pass-up  table  by  quarter  with  both 
number  and  %;show  previous  quarters  and  present  an  annual 
summary;  and 

• Track  individual  pass-up  line  performance  over  each  quarter 
of  each  fiscal  year. 

The  auditor  also  recommends  that  Service  Standard  4a  be 
coordinated  with  Service  Standard  5a  so  that  all  lines  Identified  as 
exceeding  65%  of  capacity  during  peak  periods  arc  monitored  as 
part  of  Service  Standard  4a.  Additional  data  available  through  the 
Passenger  Service  Reports  database  should  be  utilized  to  conduct 
follow-up  traffic  checks  on  lines  identified  as  having  chronic  levels  of 
pass-ups  or  peak  penod  loads  that  exceed  65%  of  capacity. 
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5a.  Peak  period  passenger  load  factors. 


Goal: 

No  groater  then  85%oJ  combined'seaiing  & standing  capacity^._  ^ ■ - 

Purpose: 

To  measure  load  factors  at  peak  periods 

jOofmition^H^ 

Moosuremont: 

jtothly  measurement  of  the  5 ()y^|.Mtronized routes  for  eaclMv|5iDn..'tSuch-chftcM  no^^ 

'l^ofteh  than  ID  weekdays  per  moiith  bn  routes  D™of>ih  '? 

most  patronized  route,  progressing  mme'  oroer’of  declinirig‘foute''patronagOTna^ingwiifurm  leas^tronized^ 
route.  The^cycle  s||BMI}6n,bejepea|ed.FY  Each  line  wiirbe  checked'twice  a year'  stfSTchecks-stfall^- 

be  conductediio  less^nMniOweekd^andwea^^  j^n^ual  checking:sch^ui^^^^  . 

estafiffs'h'ed  for  the  routes.  The  orderjivwm^^e^tK^re  checked'wiN^e^em^leQ^ 

'random  sehclion^ocess.  To'1he^e_xtenFautothated  systems  can.^substltutM^^scosftysucn!^  ' 

'the  rneasurementlTf 'anfperfortiiance  standard,  such  systems  tm^foe 

Method  of 
Moosuromont: 

Method  of  Measurement:  " Periods  of  time  includes  morning  rush  (6  a.m.  To  9 a.m.l  midday  (9  a.m.-  4 p.m.) 
evening  rush  (4  p.m.  To  7 p.m.)  and  night  (7  p.m.  To  1 a.m.).  Supervisory  personnel  shall  conduct  a one-hour,  on 
time,  and  load  standard  check  at  a maximum  load  point  at  mid-route  during  all  four  time  periods  stated  above.” 

Milostonos;  . 

'•  ....  ..V 

^oGreater  than  85^*5.  No  greater  than  85^  ' 

From  Altin/  Scrrkf  StondorOt  flrpor^  Ff20CI  end  FT2002. 
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Reported  Performance 

Muni  was  unable  to  meet  its  goal  of  no  lines  over  8S%  capacity 
In  either  FTIOOI  or  FY2002.  Due  to  changes  in  the  sampling 
methodology  between  those  two  years,  annual  performance  results 
could  not  be  accurately  compared.  This  is  discussed  further  in  the 
section  “Problems  or  Issues  Identified."  Although  the  goal  was  not 
met  in  FY2003,  substantial  improvement  was  made,  with  only  10 
lines  over  85%  capacity.  The  following  table  shows  the  number  of 
lines  that  failed  to  meet  the  standard  of  no  greater  than  85%  of 
combined  seating  and  standing  capacity  in  fiscal  years  2001,  2002, 
2003  and  the  first  and  second  quarters  of  FY2004. 


Table  A-7  Number  of  Lines  that  Exceeded  85%  Capacity, 
FY200I  -FY2004 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 
fV200l 

The  following  changes  m sampling  methodology  wore  approved  by 
the  MTA  Board  and  subsequently  implemented  in  FY200I; 

• Light  rail  (LRV)  and  cable  car  divisions  were  combined  Into 
one  rail  sampling  rotation  with  results  reported  separately; 
eight  checks  per  month  were  reduced  to  five  or  four  checks 
per  month. 

• Three  motorcoach  divisions  were  combined  into  one  motor* 
coach  sampling  rotation;  IS  checks  per  month  were  reduced 
to  five  checks  per  month. 

• Two  trolleycoach  divisions  were  combined  into  one  trolley* 
coach  sampling  rouilon;  10  checks  per  month  were  reduced 
to  five  checks  per  month. 

Lines  were  sampled  within  each  mode  according  to  ranked 
patronage  as  described  In  the  definition  of  measurement.  At  ieast 
ten  weekday  checks  were  performed  during  each  month  for  motor 
coach  and  troiley  coach  modes.  Light  rail  and  cable  car  modes  were 
in  compliance  with  the  definition  of  measurement  for  the  months 
they  were  sampled.  All  time  periods  were  sampled  in  accordance 
with  the  definition  and  method  of  measurement. 

fy2002 

The  selection  of  routes  was  coordinated  with  the  Dau  Collection 
Plan  (DCP).adoptcd  by  Muni  for  provision  of  data  for  internal  planning 
purposes  and  to  sausfy  the  annual  NTO  (and  previously  UMTA/ 
FTA  Section  IS)  ridership  reporting  The  FY200I  metliodoiogy 
for  sampling  IS  lines/chccks  per  month  was  modified  for  Pf 2002 
to  include  nine  motorcoach  lines,  three  to  four  trolleycoach  lines, 
and  two  to  three  rail  lines.  A random  selection  process  replaced 
the  ranked  ridership  rotation  process.  Specific  locations  and  lime 
periods  conformed  to  the  DCP  and  included,  as  a minimum,  the 
periods  defined  In  Proposition  E.  p*j4A-2S 
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Verification  of  Reported  Performance 
During  the  Audit  Period 

Based  on  an  audit  of  data  for  FY200I  and  FY2002,  reported  results 
match  those  submitted  by  Muni  data  staff, 

Problems  or  Issues  Identified  During  the  Audit  Period 

Although  the  total  number  of  lines  checked  was  about  the  same  for 
both  FY200I  and  FY2002,  the  number  of  lines  checked  within  each 
mode  changed  between  the  two  years.  In  FY200I. three  to  five  lines 
were  checked  monthly  In  each  of  the  three  modes  - rail,  trolley  and 
motor  coach.  In  FY2002,  monthly  checks  consisted  of  one  to  three 
rail  lines,  three  to  four  trolley  lines,  and  nine  to  ten  motor  coach 
lines.  While  this  change  led  to  a better  balance  of  sampling  between 
modes,  It  made  It  Impossible  to  accurately  compare  performance 
between  FY200I  and  FY2002. 

In  FY200 1 , 48  out  of  the  approximately  1 75  lines  checked,  or  27.4%, 
had  load  factors  that  exceeded  85%.  In  FY2002,  23  out  of  the 
approximately  176  lines  checked,  or  13.1%,  had  load  factors  that 
exceeded  85%.  Analysis  of  the  24  lines  that  improved  load  factors 
between  FY200I  and  FY2002  showed  that  there  was  a reduction 
from  46  monthly  samples  above  85%.  to  only  six  monthly  samples 
above  85%.  For  those  same  24  lines,  the  rate  of  sampling  was 
reduced  by  30.1%  between  FY200I  and  FY2002from  83  samples  to 
58  samples, 


Chonges  in  Aleosurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  has  made  no  changes  to 
the  definition  of  measurement,  method  of  measurement,  nor  the 
milestones. 


Auditor  Recommendations 


6a. 


In  all  cases  of  non*compllanc  lines,  sampling  frequency  should  b< 
Increased  from  the  previous  year  until  a trend  in  compliance  is 
noted.  This  will  enable  an  accurate  evaluation  of  improvement  in 
line  performance  from  year  to  year. 

A more  meaningful  method  of  reporting  may  be  to  capture  individui 
line  load  fectors  by  15-mlnute  periods  (quarter-hour  peak  periods) 
rather  than  for  a full  hour.  This  coincides  with  schedule  evaluation 
and  revision  analysis  processes,  and  would  enable  Muni  staff  to 
better  pinpoint  time  periods  that  require  greater  capacity  (in  order 
to  relieve  overcrowding). 

Performance  results  have  been  reported  in  two  different  formats 
The  4th  Quarter  FY200I  Service  Standards  Report  contained 
a system-wide  table  summarized  by  quarter  which  included  the 
following  information:  number  of  pass-ups,  percentage  of  pass-ups 
number  of  checks  at  MLR  and  the  lines  with  more  than  two  pass-ups 
The  4th  Quarter  FY2002  Service  Standards  Report  did  not  contain 
a cumulative  (each  quarter  and  annual)  systemwide  table.  Because 
the  sampling  plan  for  lines/modes  varies  by  month,  both  a quarterly 
and  an  annual  performance  summary  should  be  presented. 

The  auditor  also  recommends  that  Service  Standard  5a  be 
coordinated  with  Service  Standard  4a  so  that  alt  lines  identified  as 
exceeding  85%  of  capacity  during  peak  periods  are  monitored  as 
part  of  Service  Standard  4a.  Additional  data  available  through  the 
Passenger  Service  Reports  database  should  be  utilized  to  conduci 
follow-up  traffic  checks  on  lines  identified  as  having  chronic  ieveis  oi 
pass-ups  or  peak  period  loads  that  exceed  85%  of  capacity. 
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6a.  Actual  headways  against  scheduled  headways  on  all  radial,  express,  cross-town, 
secondary,  and  feeder  lines  for  peak,  base,  evening,  and  late  night  services. 


{Goal:  ' 

1- 

lOperate^no'.less  than.80%  of  the  time  within  30?t  or  10  minutes  of  scheduled  headway  (whichever  is  less). 
iFY2002:  Operate  no  less  then  85^^  of  the  time  within  30^t  or  10  minutes  of  scheduled  headway  (whichever  is 
ilhTss)V_'___.\„.  . _ 

Purpose: 

Measurement  of  scheduled  headways  against  actual  headways. 

Definition  of  j 
Measurement:  j 

i ; ■■  i 

1 ; 

1 ■ - ■ 
: 

1 . .y 

FY2G01:;Will  mea'sure  average  hea'dway  monthly  and  will  designate  type  of  line  as  part  of  Service  Standard  1 a 
of  t^les  that  run  on"time  kcording  to  published  schedules.  Measurement  of  the  S most  patronized  routes 
for  eadi'iirvIsiori'^Such  checks  shall  be  conducted  no  less  often  than  10  weekdays  and  weekends  per  month 
ion  5 different  roufes'per  month  in  each  division,  beginning  with  the  most  patronized  route,  progressing  in  the 
Vrdefof.declining  route  patronage,  and  ending  with  the  least  patronized  route.  The  cycle  shall. then  bo  repeated. 
;FY.2002;  fach.line  wjll  be  checked  tviiice  a year.  Such  checks  shall_bej:onductedno  lass  often  than  10  weekdays 
landweekends’per  check.'  An  annual  checking  schedule*  shall  be’est3blis^J^for  thB*routes.,,The  order  ih'which 
the  routes  are  ch’eckedAviirba’deterrTimed  monthly  through  a random  seleclion'ptocew^  Jo  the  extent  automated 
systems  can  be'’substituted  at  less  cost  for  such  checks,  or  the  measurement  of  any  perfoimahce  standard,  such 
svstemsinu'sl  housed; 

Method  of 
Measurement; 

Method  of  Measurement;  Check  the  headways  of  designated  lines.  'Periods  of  time  includes  morning  rush  (6 
a.m.  to  9 a.m.)  midday  (9  a.m.  • 4 p.m.)  evening  rush  (4  p.m.  to  7 p.m.)  and  night  (7p.m.  to  1 a.m,).  Supervisory 
personnel  shall  conduct  a one-hour,  on-time,  and  load  standard  check  at  a maximum  load  point  at  mid-rouio  during 
all  four  time  periods  stated  above.* 

^Milestones:  ( 

FY2001  ■ FY2002 

.80%.  85%  • 

from  Mun/  Semce  Sundords  ftep«rt  fYlOO  I and  fYJOOt 
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Reported  Performance 

For  the  fourth  quarter  of  FY200 1,  systemwide  headway  adherence 
was  56.8%  (the  percentage  of  sampled  headways  that  were  within 
30%  or  10  minutes  of  the  scheduled  headway,  whichever  was  less). 
In  the  fourth  quarter  of  FY2002,  systemwide  headway  adherence 
rose  to  72.1%,  however  results  between  FY200I  and  FY2002  are 
not  directly  comparable  due  to  a change  in  sampling  methodology 
between  FY200I  and  FY2002.  Headway  adherence  did  not  improve 
in  FY2003  or  the  first  quarter  of  FY2004.  Table  A-8  shows  the 
average  percentage  of  headway  adherence  for  the  lines  measured 
within  each  month  of  fiscal  years  200 1 , 2002, 2003  and  the  first  and 
second  quarters  of  FY2004. 


Table  A-8  Average  Percentage  of  Lines  Adhering  to 
Scheduled  Headways.  FY200 1 - FY2004 
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Figure  A- 1 Headway  Adherence 
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from  ATurU  Service  Stsndardt  RtS-oa  Quarter  fT2003. 


Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

mm 

The  following  changes  in  sampling  methodology  were  approved  b> 
the  MTA  Board  and  subsequently  implemented  In  FY200I; 

Light  rail  (LRV)  and  cable  car  divisions  were  combined  int: 
one  rail  sampling  rotation  with  results  reported  separately 
eight  checks  per  month  were  reduced  to  five  or  four  checio 
per  month. 

Three  motorcoach  divisions  were  combined  into  one  mote 
coach  sampling  rotation:  1 5 checks  per  month  were  reduced 
to  five  checks  per  month. 


Line 

patr 

ten 

coai 

in  C' 

they 

With 

FY20' 

The 
Plan 
purp 
FTA 
sami 
inck 
A ra 
proc 
and 

The 
lines 
ratic 
heaa 
itwc 
com 
the  r 
was 


Proposition  E Municipal  Transportation  Quality  Review 

• Two  trolleycoach  divisions  were  combined  into  one  trolley- 
coach  sampling  rotation;  10  checks  per  month  were  reduced 
to  five  checks  per  month. 

Lines  were  sampled  within  each  mode  according  to  ranked 
patronage  as  described  in  the  definition  of  measuremenc  At  least 
ten  weekday  checks  were  performed  during  each  month  for  motor 
coach  and  trolley  coach  modes,  Light  rail  and  cable  car  modes  were 
in  compliance  with  the  definition  of  measurement  for  the  months 
they  were  sampled.  All  time  periods  were  sampled  in  accordance 
with  the  definition  and  method  of  measurement 

Fy2002 

The  selection  of  routes  was  coordinated  with  the  Data  Collection 
Plan  (DCP).adopted  by  Muni  for  provision  of  data  for  internal  planning 
purposes  and  to  satisfy  the  annual  NTD  (and  previously  UMTA/ 
FTA  Section  15)  ridership  reporting.  The  FY200I  methodology  for 
sampling  15  lines/checks  per  month  v/as  modified  for  FY2002  to 
include  nine  motorcoach  lines  and  three  to  four  trolleycoach  lines. 
A random  selection  process  replaced  the  ranked  ridership  rotation 
process.  Specific  locations  and  time  periods  conformed  to  the  DCP 
and  included,  as  a minimum,  the  Proposition  E defined  time  periods. 

The  two  rail  modes  (light  rail  and  cable  car)  as  well  as  major  radial 
lines  were  removed  from  the  measurement  for  Pf2002.  The 
rationale  for  this  decision  was  that  the  standard  of  30%  or  bencr 
headway  adherence  was  not  applicable  to  smaller  headways  because 
it  would  not  be  meaningful.  Overall. 60  lines  were  sampled  in  FY2002, 
compared  with  82  lines  in  Pf200l.  (This  change  was  approved  by 
the  MTA  Board.)  Measurement  of  headway  adherence  for  all  modes 
was  resumed  in  Pl'2004. 
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Verification  of  Reported  Performance 
During  the  Audit  Period 

Based  on  an  audit  of  data  for  FY200I  and  FY2002.  reported  results 
match  those  submitted  by  Muni  daa  staff. 

Prob/ems  or  /ssues  Identified  During  the  Audit  Period 

Muni  has  been  unable  to  meet  its  headway  adherence  go.als  during 
all  years  of  measurement  Adjustments  to  the  method  and  definition 
of  measurement  had  some  impact  on  performance  results,  however 
changes  that  were  made  In  FY2002  and  FY2003  were  reversed  in 
FY2004  Consequently.  It  Is  necessary  to  further  examine  the  reasons 
for  failure  to  meet  performance  goals  as  well  as  the  goals  themselves, 
the  definition  of  measurement  and  the  method  of  measurement. 

Chonges  in  Measurement  Since  the  Audit  Period 

The  following  change  was  made  to  the  Definition  of  Measurement 
in  FY2004: 

Each  /me  will  be  checked  twice  o year.  Such  checks  sho//  be  conducted 
no  less  often  than  1 0 weekdays  and  weekends  per  check.  An  annual 
checking  schedule  shall  be  cswb/ished  for  the  routes.  The  order  In 
which  the  routes  are  checked  will  be  determined  monthly  through 
a random  seleaJon  process.  To  the  extent  automated  systems  can 
be  subsvtuted  at  less  cost  for  such  checks,  or  the  measurement 
of  any  perfomionce  stondord,  such  systems  must  be  used.  Actual 
headways  ogomst  scheduled  headways  on  all  radial  express,  cross- 
town, secondary,  and  feeder  lines  for  peak,  base,  evening,  and  late 
night  services.  Will  also  check  lines  during  periods  when 
their  headway  Is  greoicr  chan  10  minutes. 


San  Francisco  Municipal  Railway 


Action  Plan 

Streamlining  the  Service  Standards  and  Improving  the  Delinition  of  Heasurement 

Headway  adherence  Is  typically  used  to  measure  the  reliability  of 
transit  services  operating  at  headways  that  are  10  minutes  or  less/ 
From  a passenger's  perspective,  only  headway  adherence  matters 
when  headways  are  10  minutes  or  less,  since  passengers  do  not  rely 
on  schedules  to  plan  their  trips  because  they  can  rely  on  the  fact 
that  a vehicle  will  arrive  shortly  after  they  arrive  at  their  stop. 

Given  that  Muni  has  resumed  measuring  headway  adherence  on 
radial  lines  in  FY2004,  It  Is  important  to  examine  the  most  effective 
ways  in  which  to  measure  headway  adherence  for  high  frequency 
transit  services  (those  with  headway  intervals  of  10  minutes  or 
less).  Headway  adherence  is  a less  relevant  reliability  measure  for 
transit  services  with  headways  greater  than  1 0 minutes,  particularly 
if  service  reliability  Is  already  being  measured  by  schedule  adherence. 
Consequently,  the  auditor  recommends  that  headway  adherence  be 
measured  only  for  those  transit  services  with  headway  intervals 
of  10  minutes  or  less,  and  that  schedule  adherence  (I.e.,  on-time 
performance)  be  measured  for  all  other  transit  services  with 
headway  Intervals  that  arc  greater  than  10  minutes.  Because  these 
changes  would  require  an  amendment  to  the  City  Charter,  It  may 
not  be  practical  to  implement  them  in  the  near  term. 

We  also  recommend  that  the  method  of  measuring  headway 
adherence  be  modified  so  that  It  is  more  relevant  to  high  frequency 
service.  Foliowing  are  three  possible  alternatives  for  changes  to  the 
method  of  measurement: 


P#9pA40 
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• Alternative#!:  The  method  of  headway  adherence  mej. 
surement  recommended  in  the  Transit  Copoerty  and  Quolfti 
of  Service  Manual — 2nd  Edition  is  based  on  the  "coefftcient  of 
variation  of  headways"  of  transit  vehicles  and  is  calculated  In 
the  following  way: 

Coefficient  of  variation  of  headways  - 

(standard  deviation  of  headway  deviations)+{me3n  scheduled  headway)’ 

This  calculation  compares  the  difference  between  actual  ano 
scheduled  headways  with  that  of  the  scheduled  headway  ^ 
lower  coefficient  means  that  there  is  very  little  deviation  ir 
actual  headways  from  scheduled  headways  and  that  service 
reliability  is  excellent  A larger  coefficient  means  that  there  li 
a significant  amount  of  deviation  between  actual  and  scheduled 
headways  and  that  service  reliability  is  poor.  The  followir- 
table  summarizes  information  from  the  Tronsrt  Capacity  one 
Quality  of  Service  Manual — 2nd  Edition  regarding  coefficier 
values  and  associated  levels  of  service  (LOS).  It  is  importer  L 
to  note  that  this  measurement  and  its  associated  LOS  values  Fi 
onfy  applies  to  transit  services  with  headways  of  10  minute, 
or  less. 


■ibJi^s-4e 
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If  Muni  chooses  to  use  this  measurement  approach,  a goal  of 
LOS  C or  better  should  be  established  for  headway  adher- 
ence. 


E 


Service  is  provided  like  clockwork,  wiih  very  regular 
headways. 


0.22  0,30 


Most  vehicles  are  off  the  scheduled  headway  by  a few 
minutes,  but  the  likelihood  of  being  off-headway  by  more 
than  one-half  the  scheduled  headway  amount  Is  low  |e.g„ 
5 minutes  off  a 10  minute  scheduled  headway). 


0.31-0.39 


Vehicles  are  often  off-headway,  with  a few  headways 
much  longer  or  shorter  than  scheduled.  


0.40-0.52 


Headways  are  quite  Irregular,  with  up  to  one  in  three 
vehicles  one-half  a headway  or  more  olt-headway. 


0.53-0.74 


Bunching  occurs  frequently. 


F 


0.75 


Most  vehicles  are  bunched. 


From  the  Tnznirr  CepcdtfOnd  QiwCtyefServKcManuo^Ind  EMien  f2003l- 


• Alternative  #2:  The  approach  used  by  MTA-NewYork  City 
Transit  is  similar  to  that  currently  used  by  Mum,  but  has  a 
specified  time  window  and  focuses  solely  on  incidences  where 
headway  times  are  exceeded.  MTA-New  York  City  Transit 
measures  the  percentage  of  transit  vehicle  arrivals  where  the 
actual  headway  exceeded  the  scheduled  headway  by  more 
than  3 minutes.  Actual  headways  that  are  less  than  scheduled 
headways  are  not  considered,  based  on  the  assumption  that 
short  headways  generally  result  from  the  previous  vehicle's 
long  headway,  and  that  the  vehicle  with  the  long  headway  is 
the  one  that  affects  passenger  service  quality,  as  a result  of 
longer  wait  times  and  more  crowded  conditions  on  board.' 


* Ibid.  (1 3-49 
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♦ Alternative  #3:  This  alternative  should  only  be  used  if  Muni 
continues  to  measure  schedule  adherence  for  all  lincs.rcgard- 
less  of  their  headviray  interval.  While  radial  lines'  frequency 
of  scheduled  service  accommodates  passenger  demand,  the 
reality  of  service  delivery  often  presents  gapping  and  bunching, 
resulting  in  abnormal  headways.  A surrogate  measurement  to 
capture  the  provision  of  capacity  on  major  radial  lines  is  the 
measurement  of  throughput.  This  observation  and  measure- 
ment of  actual  trips  (the  number  of  scheduled  vehicles  that 
pass  a particular  point  within  a given  period  of  time)  allows 
for  a consistent  report  on  trunk  capacity  and  street  condi- 
tions. Both  Service  Standards  la  (on-time  performance)  and 
5a  {peak  period  passenger  load  factors)  measure  components 
of  major  radial  service  delivery,  but  ncitlicr  captures  loss  of 
capacity.  MTA  could  consider  substituting  tliroughput  for 
Service  Standard  6a.The  Proposition  E urget  of  85%  should 
be  modified  over  a multi-year  program  to  reach  90%,  The 
correlation  of  this  target  to  that  of  Service  Standard  2a  ("not 
less  than  98.5  % on  July  1 , 2004")  is  important,  so  that  there 
is  an  allowance  for  on-sircct  loss  of  specific  trips/throughpui 
due  to  equipment  malfunctions,  blockages,  etc. 

Additional  Recommendations  to  Improve  feriormance 

As  was  the  case  with  on-time  performance  in  Service  Sundard  la. 
it  is  likely  that  on-street  attention  to  supervision  and  prcp.aration  of 
new  schedules  will  be  needed  in  order  to  achieve  headw,ay  adherence 
goals.  For  further  detail.picasc  sec  the  recommendations  made  with 
regard  to"Schcdule  Evaluaiion/Rcmcdiation"  and"Routc  Evaluation" 
under  the  Action  Plan  for  Service  Standard  la. 
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Samplin|  fle5ources  - Reallocation  of  Terminal  Checking  to  Inaeaied  NTD 
Checking 

Although  the  Proposition  E measurement  requirement  included 
chocks  at  tcrmlnalsand  established  timepoints.the  checking  resources 
expended  to  collect  from  terminals  Is  about  twice  that  required  for 
established/lntermedlate  timepoints,  The  Proposition  E mid-point 
sampling  plans  arc  coordinated  with  Muni’s  NTD  sampling  plans  to 
minimize  the  checking  resources  expended.  The  Data  Collection 
Plan  for  FY2003  (August  1 . 2002)  Indicates  that  9,720  hours  per  year 
of  checking  Is  required  for  the  Proposition  E program  at  4.5  locations 
per  line.  Additionally,  3,432  hours  per  year  of  checking  is  required 
for  the  NTD  program  at  two  locations  per  line.  Clearly,  a significant 
74%  of  the  combined  programs  is  allocated  to  Proposition  E. 

The  resulting  difference  In  schedule  adherence  (between  terminals 
and  intermediate  timepoints)  Is  only  about  I %,  however.  Thus,  while 
checking  terminals  provides  a validation  of  the  checks  at  intermediate 
timepoints  and  Identifies  trips  that  leave  the  terminal  off-schedule, 
actual  results  indicate  that  such  a validation  may  not  be  necessary, 
It  would  appear  prudent  to  Increase  the  sampling  of  intermediate 
timepoint  checks  In  lieu  of  the  terminal  checks  with  the  goal  of 
sampling  each  line  each  quarter  rather  than  the  current  twice  yearly 
target. 

The  auditor  recommends  that  the  portion  of  Proposition  E checking 
hours  allocated  to  terminal  checks  (roughly  4,320  hours  per  year) 
be  reallocated  to  mid-point  locations.  The  goal  of  such  a reallocation 
would  be  to  Increase  the  frequency  of  checks  to  provide  additional 
scasonal/quartcriy  observations  on  each  mode.  Poorly  performing 
routes  should  be  targeted  for  additional  observations  as  well.  If  the 
approximately  4,320  hours  currently  allocated  to  terminal  checks 
were  reallocated  to  the  current  NTD  program  of  3.432  hours,  It 
would  more  than  double  the  frequency  of  checks  made  under  the 
NTD  program. 


Another  opportunity  for  reallocating  on-time  performance  checkmj- 
resources  could  be  found  by  caking  advantage  of  existing  technology 
On-time  performance  of  routes  that  are  monitored  by  Next  Bus 
and/or  by  the  ATCS  system  in  the  subway  could  be  done  wit^ 
improved  accuracy  by  taking  advantage  of  those  automated  systemi 
Checker  hours  deployed  on  those  routes  could  be  used  for  additioric 
monitoring  of  targeted  services. 

Coordination  with  other  Goals  and  Objectives 

On-time  performance  is  closely  related  to  the  other  performancs 
goals  within  this  group  of  service  standards.  The  steps  taken  tc 
Improve  on-time  performance  should  be  closely  coordinated  with 
actions  taken  to  achieve  other  performance  measures,  particularly 
those  related  to  schedule  adherence  (la),  delivery  of  schedule: 
service  (2a)  and  safety  (6d). 
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7o.  % vehicle  availability  and  reliability  (mean  distance  between  failure)  by 

mode. 


[Goal: 

INo  less^than  98.5%».Vehide  Availability 

. • 

Purpose: 

To  measure  the  percentage  of  equipment  available  for  service. 

'Measurement:  ^ 

IjiilohtFljrlTieasurementof  ai/ailability  as  a percentage  of  vehicles  at  each  facility’ ' 
!^ai!able  at  7’a‘.m.,and'4  p.m.  on'non-holiday .weekdays  against  peak  dern'and 

^requirement.  An'''d'9dit[Dn8l  hie8Surement  is  added  which  shows  the  number  of 
iBays  when  a facflil^mode]  'does  not  have'100%  availability. 

Method  of 
Measurement: 

The  Vehicle  Maintenance  System  provides  the  data  for  all  modes  with  the 
exception  of  Cable  Car.  A vehicle  is  considered  available  for  service  if  it  is  availablo 
for  assignment  to  an  operator  no  later  than  7:00  a.m.  and  4:00  p.m. 

[p}liiestones;  i 

;98;Wor.bothF,Y2001  and  FY  2002)  . '-v 

From  WunJ  Struct  Sundonii  Kepen,  fY200l  o»4  fY2002 
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Reported  Performance 

For  both  FY200 1 and  FY2002.  all  modes,  with  the  exception  of  light 
rail,  met  the  goal  of  98,5%  vehicle  availability  for  both  AM  and  PM 
periods.  {For  light  rail,  percent  vehicle  availability  was  97,5%  for  AM 
and  97.4%  for  PM  for  the  fourth  quarter  of  FY2002.)  The  following 
tables  summarize  the  reported  performance  for  FY200I  - FY2004. 
Vehicle  reliability  (mean  distance  between  failure)  by  mode  is  not 
reported  as  part  of  this  service  standard,  but  Is  Instead  reported 
under  standard  9a,  "Miles  between  roadcalls  by  mode.” 

Although  cumulative  performance  over  the  entire  fiscal  year  was 
reported  for  FY200I,  only  quarterly  performance  was  reported  for 
FY2002.  Average  annual  performance  was  used  to  determine  if  the 
FY200I  goal  was  met.  while  fourth  quarter  performance  was  used 
to  determine  whether  or  not  the  FY2002  goal  had  been  met  (The 
MTA  Board  established  that  performance  for  the  entire  fiscal  year 
would  be  measured  by  fourth  quarter  performance,  rather  than  by 
performance  over  the  entire  year.) To  compare  performance  for  the 
two  years,  and  subsequent  years,  only  fourth  quarter  performance 
is  presented  here.  Light  rail  appears  to  be  consistent  In  its  Inability 
to  meet  the  established  goal  during  the  fourth  quarter  of  each  fiscal 
year  for  which  data  are  available.  Other  modes  and  divisions  have 
met  the  goal  relatively  consistently. 

It  should  be  noted  that  in  many  cases  a surplus  of  vehicles  (more  than 
100%)  are  available  for  service.  However,  for  the  purposes  of  the 
Service  Standards  Report,  surplus  availability  was  simply  reported 
as  100%. 

The  definition  of  measurement  states.  "An  additional  measurement 
is  added  which  shows  the  number  of  days  when  a fccility  (mode) 
docs  not  have  100%  availability," This  measurement  was  not  included 
in  either  the  FY200I  or  FY2002  Service  Standards  Reports. 
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Table  A*9  FY200 1 AM/PM  Vehicle  Availability 


' \Vahlele^ 
eiAvallebllity^ 

IBAM 

-IstQ 
' PM^ 

?3nl(l,> 

mtm 

4tAMS 

It* 
4th  0 
uPW* 

feuij 

$0Aia 

Moior  Coach 

99.2% 

99.9% 

94.8% 

96.9% 

98.4% 

99.1% 

99.7% 

99.4% 

Flytifl.Aflic 

99.8% 

99.8% 

94% 

84.3% 

96.3% 

95.7% 

99.7% 

99.2% 

IGiklind 

97.9% 

100% 

90.4% 

96.4% 

96.6% 

100% 

98.9% 

98.4% 

Woods 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

Trollgy  Cooch 

99.8S 

99.4% 

99.7% 

97.9% 

99.9% 

99.5% 

99.6% 

90.8% 

Polf*ro-Anie 

99.5% 

86.3% 

99.7% 

34% 

99.3% 

97.4% 

98.9S 

95.0% 

Polrcfo 

100% 

100% 

100% 

99.9% 

100% 

99.9% 

99.9% 

972% 

98.5S 

Pruidio 

100% 

100% 

99.4% 

99.7% 

100% 

100% 

100% 

100% 

Light  Rill 

99.6% 

99.4% 

96.9% 

96% 

96.2% 

92.5% 

99.9% 

99.5% 

Sroda* 

98.9% 

99.4% 

99.2% 

99% 

- 

Bo  ting* 

94.6% 

92.8% 

90.7% 

652% 

- 

. 

Flint 

98.7% 

99.2% 

99.4% 

99.1% 

100% 

100% 

100% 

100% 

Cable  Car 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

From  Alvrw  Stmet  Siandanls  Report.  Quarter  mOOl 


•Note.  Rejyla  for  Bredj  end  Boeln*  divliteni  were  not  reported  leparately  eftcr  the  second  qviner 
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Table  A- 1 0 FY2002  AM/PM  Vehicle  Availability 


' Vehicle  •'  ' 
Availability]^' 

Ipat  q' 
iAM,-, 

?1«Q 

'pM* 

2nd0*. 

- am'* 

2n^L0 

'If?? 

Mt 

tpMi*- 

4lhQ 

AM 

4th  Q’ 
PM  ■ 

FY02 

GOAL 

Motor  Coach 

98.3:y 

99.1% 

96.3% 

97.?% 

99.1% 

99.9% 

99.9% 

99.6% 

ZZ£E 

FlYUfi'Ame 

97.6% 

97.2% 

100% 

100% 

69.9% 

100% 

100% 

100% 

KifUand 

95.3% 

98.7% 

932% 

91.2% 

92.6% 

97.4% 

99,7% 

99.5% 

Woods 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

98.5 

Trolley  Coach 

99.9% 

98.8% 

99.0% 

97.7% 

99.5% 

98.9% 

992% 

96.6% 

PomfO'Artic 

99.3% 

98.5% 

98.8% 

96.7% 

98.7% 

96.6S 

100% 

100% 

Poinro 

99.9% 

96.es 

99.6% 

86.8% 

99.0% 

97.7% 

100% 

99.7% 

Prasido 

100% 

lOOS 

982% 

98.5% 

99.2% 

99.6% 

96.9% 

972% 

light  Rail 

100% 

992% 

99.3% 

99.0% 

97.3% 

97.1% 

97.5% 

97.4% 

FLIna 

100% 

100% 

lOOS 

100% 

100% 

100% 

100% 

lOOS 

Cable  Car 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

f 

F 


F 


U 

h 

Cl 


From  Atuw  Sertre  Si«>daft»  Report.  4th  Quarter  FT7002. 
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Table  A-1 1 FY2003  AM/PMVehicle  Availability 


1 Vehielllil 
litvsilabTi^S 

1 

pT-' 

m 

IE? 

••AM  . 

•4ilia 
S*  PM^i 

Motor  C«tcti 

99.2S 

99.4% 

99.5% 

99.871 

99.9% 

99.9% 

99.?% 

89.6% 

PtFnn-Artic 

99.8S 

99.8% 

100% 

99.9% 

99.9% 

99.9% 

88.9S 

96.2% 

Kikland 

97.6% 

99.5% 

100% 

1D0% 

100% 

99.9% 

99.9% 

88.2% 

Woods 

100% 

99.1% 

99.9S 

99.7% 

100% 

100% 

100% 

100% 

Trolley  Coach 

99.9% 

99.1% 

I0D% 

100% 

99.9% 

99.0% 

99.9% 

99.6% 

Poirero- Artie 

99.5% 

99.3% 

100% 

100% 

100% 

100% 

99.4% 

98.4% 

Poirero 

100% 

97,7% 

lOOS 

100% 

100% 

100% 

lOOS 

100% 

Presidio 

100% 

99.9% 

100% 

100% 

99.9% 

99.6% 

100% 

100% 

LiBhtRsIl 

96.8% 

96.6% 

99.3% 

90.9% 

99.2% 

98.6% 

99.6% 

97.8S 

hlini 

100% 

106% 

98.1% 

99.3S 

99.8% 

99.7% 

100% 

100% 

Cable  Car 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

From  Muni  Service  Stondonrs  Quoner  FT  2003. 

D>Q  Kav«  not  been  eudned. 
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Table  A- 1 2 FY2004  1st  and  2nd  Quarter  AM/PM 
Vehicle  Availability 


^FVehietnS 

Itvailabililv ' 

aisio:. 

^PM>^ 

•SpAM-r 

5^01  • 
KGOAl 

Motor  Coach 

99.8% 

99.3% 

99.9% 

99.9% 

08.SS 

Ftynfi-Artic 

99.4% 

97.2% 

99.9% 

99.9% 

Kirkland 

100% 

100% 

99  9% 

99.9S 

Woods 

100% 

lOOS 

lOOS 

100% 

Trolley  Coech 

99.9% 

99.9% 

99.9% 

99.9% 

Potrera  Artie 

99.4% 

99.3% 

99.6% 

100% 

Potrero 

100% 

100% 

99.9% 

99.5% 

Presidio 

100% 

100% 

100% 

100% 

Light  Rail 

92.3% 

92.3% 

99.6% 

95.2% 

F-Line 

99.6% 

99.6% 

97.7% 

98.3% 

Cable  Car 

100% 

100% 

100% 

100% 

From  Murk  Service  Sundordi  A<F>ort,  Iti  and  2nd  Quarter  FT2004 
Dea  hive  not  been  audited. 
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Verification  of  Definition  and 

Method  of  Measurement  During  the  Audit  Period 

For  both  FY200I  and  FY2002,  data  were  collected  and  aggregated 
according  to  the  definition  and  method  of  measurement  included 
in  the  Service  Standards  Reports.  Vehicles  arc  unavailable  during 
repairs  or  maintenance  and  arc  considered  to  be  "on  hold"  if  they 
arc  unavailable  for  service  for  longer  than  a one-hour  period  of  time. 
When  repairs  or  maintenance  arc  completed,  the  vehicle  is  taken 
off  hold.  All  vehicle  maintenance  data  arc  entered  Into  the  Vehicle 
Maintenance  System  (VMS).  Operations  personnel  arc  instructed 
to  save  the  hold  count  in  VMS  each  morning  and  afternoon,  VMS 
automatically  calculates  the  dally  pcrccnngc  availability  using  this 
information. 

Daily  percenuge  availability  dau  viras  then  downlo.idcd  from  VMS 
and  entered  into  spreadsheet  software  in  order  to  calculate  monthly 
and  quarterly  availability  for  each  division  and  mode.  To  calculate 
the  overall  total  vehicle  availability  for  the  motor  coach  and  trolley 
modes,  a weighted  average  based  on  the  number  of  vehicles  that 
were  within  each  motor  coach  and  trolley  division  was  used 

Verification  of  Reported  Performance 
During  the  Audit  Period 

Based  on  an  audit  of  the  monthly  and  quarterly  VMS  reports  for  the 
AM  percent  vehicle  availability  dau  for  FY2002,  dau  appear  to  be 
reported  correctly. 
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Problems  or  Issues  Identified  During  the  Audit  Period 

No  Issues  or  problems  were  found  with  the  method  of  measurement, 
however  several  problems  with  both  reported  performance  and 
the  definition  of  measurement  were  found.  Reported  performance 
was  not  consistent  between  quarterly  Service  Standards  Reports 
In  FY200I.  For  FY200I,  first  quarter  performance  reported  in  the 
1st  Quarter  FY200I  Service  Standards  Report  did  not  match  first 
quarter  performance  reported  in  the  2nd  Quarter  FY200I  Service 
Standards  Reporc  Similarly,  third  quarter  performance  reported  in 
the  3rd  Quarter  FY200I  Service  Standards  Report  did  not  match 
reported  third  quarter  performance  in  the  4th  Quarter  FY200I 
Service  Standards  Report.  For  FY2002,  performance  was  reported 
consistently  between  quarters. 

For  both  FY200I  and  FY2002,the  definition  of  measurement  was  not 
followed  completely.  Part  of  the  definition  of  measurement  states, 
“An  additional  measurement  is  added  which  shows  the  number  of 
days  when  a facility  (mode)  does  not  have  100%  availability."  This 
Information  was  not  reported  In  either  the  FY200I  or  the  FY2002 
quarterly  Service  Standards  Reports.  While  this  statement  was 
removed  from  the  FY2003  Service  Standards  Reports.lt  was  included 
but  not  reported  In  the  FY2004  First  and  Second  Quarter  reports, 


Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  has  made  no  changes  to 
the  definition  of  measurement,  method  of  measurement,  nor  the 
milestones. 

Auditor  Recommendations 

It  is  difficult  to  ascertain  from  this  measure  whether  failures  to  meer 
98.5%  or  100%  vehicle  availability  are  due  to  an  insufficient  number 
of  spare  vehicles,  or  an  overwhelming  number  of  mechanical  failures, 
or  a combination  of  both.  Reporting  information  on  spare  ratios  (the 
number  of  spare  vehicles  divided  by  the  fleet  size)  for  each  mode 
and  division  could  provide  useful  information  about  the  efficiency  of 
fleet  utilization  and  the  underlying  causes  of  failures  to  meet  goals 
for  percent  vehicle  availability. 

The  definition  of  measurement  should  be  amended  to  remove 
the  statement,  "An  additional  measurement  is  added  which  shows 
the  number  of  days  when  a facility  (mode)  does  not  have  lOO^c 
availability,  or  this  Information  should  be  included  in  the  Service 
Standard  Reports. 
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8a.  Unscheduled  absences  by  operator,  mechanical  and 
administrative  personnel. 


Goal: 


Purpose; 


Definition  of 
Measurement: 


jAnfui3l5%,reduction  of  YTO  for  Maintenance  IMechanical),  ,10%' reduction  for  Transportation 
Operators),  and'5%  jeduction  for  Administration.  • ' ■ ’ 


To  measure  unscheduled  absences. 


■Monthly  measurement  of  unscheduled  absences  is  a defined  as  time  that  is  not  scheduled  in 
a'dvance'  and  includes  the  following  payroll  categories: 

,Sick  pay  (with'payh'Sickleave  (without  pay).  AWOL,  Worker's  Comp,  SDl.dong-torm  illnoss), 
aWAssauVt.tS^  j ' :k.  . ' 


.-'in- 


Method  of 
Measurement: 


The  online  TESS  and  the  online  Attendance  Tracking  System  currently  provides  the  data  as  a 
calculation  of  scheduled  hours  available  against  unscheduled  hours.  Unscheduled  absoncos  is 
tracked  for  operators,  mechanical  and  admtnistrative  staff  by  mode. 


'Milestones: 


FY2001 


FY2002 


% 


mhsihbi 

Maintenance 

5%  Reduction  of  FYTD 

Maintenance 

5%  Reduction  of  FYTD 

Operations 

10%  Reduction  of 
FYTO 

Administration 

5%  Reduction  of  FYTD 

Administration 

5%  Reduction  of  FYTO 

! Operations 

5%  Reduction  of  FYTO 

Transit  Operators 

10S  Reduction  of 
FYTD 

Ffwn  Muni  Snma  Stantfsrti  0700 1 and  0700i 


Pipe  AO  r 
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Reported  Performance 

Unscheduled  absence  hours  were  reported  as  apercentage  of  the  total 
scheduled  hours  for  each  division.  For  FY200 1 , Operations  absences 
Included  both  Transit  Operators  as  well  as  other  Operations  support 
staff.  In  FY2002,  these  two  divisions  were  reported  separately,  and 
different  milestones  were  established  for  each. 

As  shown  in  the  following  table,  all  three  divisions  failed  to  meet  their 
percentage  goals  for  FY200I.  While  the  Administration  Division 
failed  to  meet  its  established  goal  again  In  FY2002.  all  other  divisions 
either  met  or  did  better  than  their  established  goals  In  FY2002. 
All  divisions  were  able  to  meet  their  goals  in  FY2003.  Table  A*  1 3 
summarizes  the  percentage  of  unscheduled  absences  for  FY200I 
- FY2004. 


Table  A- 1 3 Percentage  of  Unscheduled  Absences, 
FY200I-FY2004 


From  MuN  Sianctonft  Quatttt  mOOl.FnW.  PQOO]  ayj  2nd  Qudrtef  fY2004. 

Old  In  IqIio  hrr«  net  bMn  audited 

‘Not*  ThintfWQOon  IncIgiM  both  Op*rtt»oni  andTpinUi  O^ton  In  FYIOOI 


Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Scheduled  and  unscheduled  hours  are  tracked  in  two  automated 
timekeeping  systems  - ATS  and  TESS.  TESS  is  a relatively  new 
timekeeping  system,  and  hours  for  maintenance,  administrative  and 
operations  personnel  Is  tracked  In  this  system.  However,  some 
hours  for  maintenance  and  operations  arc  still  tracked  in  ATS.  and  all 
transit  operator  hours  arc  tracked  in  ATS. 

P»0eA-ad 


Verification  of  Reported  Performance  During  the 
Audit  Period 

Based  on  an  audit  of  data  for  the  third  quarter  of  FY2002,  all  data 
appear  to  be  reported  correctly. 

Problems  or  Issues  Identified  During  the  Audit  Period 

There  were  no  problems  or  issues  with  regard  to  reported 
performance,  or  the  definition  or  method  of  measurement 
Tabulating  unscheduled  and  scheduled  hours  data  from  two  different 
timekeeping  systems  is  time-consuming,  however,  and  would  be 
simplified  if  all  timekeeping  was  done  within  one  database. 

Chonges  »n  Aleosurement  5/nce  the  Auc/it  Period 

The  Method  of  Measurement  was  changed  in  FY2003  to: 

The  online  Attendance  Tracking  System  provides  the  data  as  a 
cokuladon  of  scheduled  hours  available  against  unscheduled  hours 
for  Transit  Operators.  The  TESS  Timekeeping  System  provides 
the  unscheduled  absences  for  Maintenance,  Administrative  and 
Operations  employees.  Unscheduled  absences  ore  tracked  for 
operators,  mechanical  and  adminisvative  staff  by  mode. 

In  FY2004,  the  Method  of  Measurement  was  changed  back  to  the 
original  language. 


Auditor  Recommendations 

in  terms  of  service  reliability.only  a significant  number  of  unscheduled 
absences  of  Maintenance  and  Operations  staff  will  negatively  impact 
service  provision.  Additionally,  the  percentage  of  unscheduled 
absences  for  the  Administration  Division  is  the  lowest  for  ail  divisions 
measured.  Consequently,  the  goal  for  reduction  of  unscheduled 
absences  for  the  Administration  Division  should  be  relaxed,  and  a 
more  reasonable  target  should  be  established. 
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providing  new  information  about  the  underlying  causes  for 
unscheduled  absences,  if  possible,  would  be  useful  to  both  Muni 
management  and  the  public. 
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9a.  Miles  between  roadcalls  by  mode. 


iGoalr  't 

llncieasB'ihe’mjles  between  ^03dc^lls;;^ 

Purposo: 

Measure  reliability  through  the  miles  a vehicle  travels  between  failures, 

Qolinltion  of- 
Moasuromont: 

1 

■ ■ ^ 

[MonihlVJlfeasuremGnl  is>tijf^ntlydictated  b^he  Fe'deral  Transit  , 

• FailSfe  classified  a j^her  a major  or»  failufe^Jjai^ejernejjJo^^ 
■miBm.Tor.eachincidqn^^^  a major  or  mlnoSfallure.je^ 

)trip%tMfel^I^  If'theifeilureoc^^dU^^ 

plitcTiTdethisirireveiiue-.vehiclBsvstemfailutes..'',; — . v. 

Mothod  ot 
Moosuromcnt: 

Data  Is  collected  and  input  into  the  online  Vehicle  Maintenance  System  for  all  revenue  vehicles 
except  for  Cable  Car,  which  has  its  own  internal  tracking  system.  Reports  are  generated  and 
the  data  for  roadcalls  is  analyzed  against  the  Central  Control  tog.  All  verifiable  major  and  minor 
mechanical  defects  are  included  as  part  of  the  Mean  Distance  between  defects  number.  Areas 
that  do  not  result  in  a chargeable  roadcall  to  the  maintenance  shops  include  accidents,  sick 
oessenqers.  vandalism,  body  damage  and  broken  windows. 

Mitostonos; 

'Milestones:  Increase  miles  between  failures^f^'' ^ \ 'i' ' 

fntn  MvnI  SenVc  Slondon)!  FT700I  end  FT7002 
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Reported  Performance 

Although  cumulative  performance  over  the  entire  fiscal  year  was 
reported  for  FY200 1 , only  quarterly  performance  was  reported  for 
FY2002,  and  fourth  quarter  performance  was  used  to  determine 
whether  the  FY2002  goal  had  been  met  (The  MTA  Board  established 
that  performance  for  the  entire  fiscal  year  would  be  measured  by 
fourth  quarter  performance,  rather  than  cumulatively.) 

In  FY200I,  the  Kirkland  motor  coach,  Sondard  Potrero,  Presidio. 
Boeing  Light  Rail.  Breda  Light  Rail  and  Cable  Car  divisions  all  met 
their  fiscal  year  goals  at  the  end  of  the  fourth  quarter.  In  FY2002. 
the  Flynn-Articulated,  Presidio,  Boeing  Light  Rail.  Breda  Light  Rail. 


and  Cable  Car  divisions  all  met  their  fiscal  year  goals  at  the  end  of 
the  fourth  quarter.  In  FY2003,  most  divisions  were  able  to  meet 
their  goals  during  the  fourth  quarter,  however  fewer  were  able  to 
meet  their  goals  with  regard  to  cumulative  annual  performance. 
In  FY2004.  only  the  Cable  Car  and  Flynn-Articuhted  divisions  are 
close  to  meeting  their  goals  (as  of  the  second  quarter).  Table  A- 1 4 
summarizes  this  information. 


Table  A-14  Miles  between  Roadcails  (Mean  Distance  between  Failures),  FY200I  - FY2004 


[ 


FY2001 


FY2002 


FY2003 


FY2004 


1 Mode  and  Division 

4th  Quarter 

FYTO 

Goal 

4th  Qoartar 

Goal 

4th  Quartar 

FYTO 

Goal 

2nd  Quartar 

Goal 

Bilotor-Coach  - Mean  Distance  Between'FaiflTFes^l 

Miles).  i 

Flynn-Artic 

837 

873 

1,500' 

1,929 

1.500 

3.033 

2.299 

1.750 

1.998 

2.000 

Woods 

1.773 

1,902 

3,000; 

1,760 

3,000 

2.802 

2.176 

2.500 

2.391 

2.750 

Kirkland 

3,467 1 

3,138 

3,000 

2,381 

3.000 

3.992 

2.918 

3.000 

2.736 

3.000 

BlIISVl  Coach  -'Mnafi  f 

listance.Between  Failures  1 

Miles)  1 

Artie  Potrero 

443 

509 

500 

508 

750 

583 

541 

650 

629 

675 

Standard  Potrero 

691 

737 

550 

795 

eoo 

86! 

762 

1.000 

812 

1.000 

Presidio 

1,375 

1.002 

550 

1.223 

BOO 

1.22! 

1.279 

1.000 

1.103\ 

1.250 

Bail  - Mean  Distance  £ 

letween  Failures  (Miles) 

Boeing  Light  Rail 

1,706 

1,447 

1,300 

Fleet  Retired  | 

Breda  Light  Rail 

3,271 

2,246 

1,800 

3,276 

1.900 

3.128 

3328 

3000\ 

3.128 

3.500 

PCC  (F-Linel 

608 

90S 

1,250 

1,496 

1,250 

1.148 

1.309 

1.250  \ 

930 

1.250 

IGable  Caf - Mean  Distance  Between  Failures  (Miles)..  i 

L 

' 5,620 

4,809 

2,530 

5.631 

3.000 

6.387\ 

5.658' 

5.000. 

5.456, 

5.500 

From  Mum  Sertic*  Swntfortj  Quoftef  fY200I.Fr2002.Fr2001  ana QvarurfT2004. 

Oia  In  iBlJa  hzv«  not  betn  audited 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Mean  distance  between  failures  (MDBF).  or  miles  between  roadcalls, 
data  were  collected  and  tabulated  according  to  the  established 
definition  and  method  of  measurement,  however  roadcall  data  were 
tabulated  directly  from  the  Central  Control  log  and  mileage  data 
were  tabulated  from  the  Vehicle  Maintenance  System  (VMS)  for  all 
modes  except  cable  car.  Following  Is  a more  detailed  description  of 
data  collection  and  aggregation  procedures  used  for  each  division. 
All  divisions  used  the  number  of  FTA-reportable  roadcalls  (ma|or  or 
minor  mechanical  failures)  to  determine  MDBF. 

Quarterly  MDBF  represents  the  average  MDBF  data  for  each  month 
within  the  quarter,  except  for  LRV.  Most  LRV  quarterly  calculations 
were  made  by  dividing  the  total  mileage  for  the  quarter  by  the  total 
number  of  roadcalls.  rather  than  averaging  the  MDBF  for  each  month 
within  the  quarter. 

Motor  Coach 

Prior  to  October  2001.  roadcalls  were  logged  directly  with  the 
Motorcoach  division.  During  this  time,  the  number  of  reported 
roadcalls  entered  Into  VMS  was  not  checked  against  the  Central 
Control  log,  and  it  was  likely  that  the  number  of  roadcalls  entered 
Into  VMS  was  slightly  less  than  the  actual  number  of  roadcalls  as  a 
result  After  October  2001.  roadcall  data  were  downloaded  from 
the  Central  Control  log  on  a dally  basis.  The  Central  Control  log 
Nvas  compared  to  the  repair  log  to  determine  the  actual  number 
of  failures,  (Occasionally,  reported  failures  were  determined  to  be 
Inaccurate  after  maintenance  soff  found  no  mechanical  defects.) 
Spreadsheet  software  was  used  to  tabulate  the  number  of  roadcalls 
and  MDBF  for  each  quarter  during  FY200I  and  FY2002. 


Pa0oA4: 


Vehicle  mileage  was  also  tracked  using  VMS.  All  vehicles  used  hub 
odometers, and  mileage  counts  were  electronically  uploaded  directl; 
Into  VMS.  Mileage  data  was  double  checked  against  fuel  usage  dao 
to  determine  any  Inaccuracies  or  mistakes. 

Polrero  Trolley 

Roadcall  data  were  downloaded  from  the  Central  Control  log  and 
tabulated  either  by  hand  or  using  spreadsheet  software.  Mileage 
data  obtained  from  either  dash  or  hub  odometers  were  enterec 
Into  VMS  and  double  checked  by  maintenance  staff.  Spreadsheet 
software  was  used  to  calculate  MDBF  using  roadcall  data  from  the 
Central  Control  log  and  mileage  data  fromVMS. 

Presidio  Trolley 

Prior  to  March  2001,  roadcall  data  was  obtained  daily  from  the 
Central  Control  log  and  was  totaled  for  each  month  and  quarter 
After  March  2001,  the  division  used  its  own  daily  roadcall  log  to 
determine  the  number  of  roadcalls  or  failures. 

Maintenance  staff  recorded  mileage  daily  on  a handwritten  log  and 
then  entered  It  into  VMS.  A supervisor  double-checked  all  mileage 
entries.  MDBF  was  calculated  monthly  by  hand  using  totals  from 
printouts  of  the  Central  Control  log  (for  roadcalls)  and  VMS  (for 
mileage). 


Proposillon  E Municipal  Transportation  Quality  Review 
Light  Rail  Vehicles 

Roadcalls  were  recorded  by  maintenance  personnel.  Only  roadcalls 
that  required  more  than  five  minutes  to  address  (from  arrival  to 
resolution)  and  were  FTA-reportable  were  counted  toward  the 
MDBF  calculation.  In  PflOO  I and  FY2002,  LRV  roadcalls  did  not  go 
through  Central  Control,  but  were  tracked  by  the  division.  The  PCC 

(maintenance  crew  also  tracked  their  roadcalls.  Roadcall  data  from 
PCC  and  LRV  maintenance  was  cross-checked  for  any  duplications, 
and  dien  tabulated. 

'For  light  rail  vehicles  other  than  historic  PCC  cars,  mileage 
♦was  tracked  daily  and  then  entered  into  VMS.  For  historic  PCC 
cars  (which  do  not  have  odometers),  staff  calculated  mileage  by 
multiplying  the  number  of  service  hours  by  9.5  miles  per  hour 
(which  is  assumed  to  be  the  average  travel  speed  for  these  vehicles). 
Staff  used  spreadsheet  software  to  calculate  MDBF  from  reported 
roadcall  and  mileage  data. 

Cable  Car 

Because  cable  cars  do  not  have  odometers,  staff  calculated  the  actual 
fleet  mileage  by  subtracting  the  number  of  miles  lost  due  to  missed 
trips  and  chargeable  roadcalls  from  the  scheduled  mileage  (scheduled 
trips  per  day  multiplied  by  the  miles  per  trip)  for  each  day. 

Roadcalls  from  the  roadcall  log  were  recorded  into  the  daily  morning 
report.  Daily  reports  were  used  to  calculate  the  number  of  roadcalls 
per  month.  MDBF  calculations  for  each  month  were  made  by  hand 
using  the  calculated  mileage  and  roadcall  data. 
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Verification  of  Reported  Performance  During  the 
Audit  Period 


Motor  Coach 

Based  on  an  audit  of  FY200I  data,  only  the  reported  performance 
data  from  the  third  quarter  matched  the  original  data  provided  by 
the  division.  For  the  first,  second  and  fourth  quarters  of  FY200I. 
source  data  did  not  match  what  was  reported  in  the  4th  Quarter 
FY200 1 Service  Standards  Rcporc  The  following  discrepancies  were 
found: 


Potrero  Trolley 

Based  on  an  audit  of  data  for  the  first  and  fourth  quarters  of  FT2002 
for  both  the  Standard  and  Artie  Divisions,  data  appear  to  be  reported 
correctly. 

Presidio  Trolley 

Based  on  an  audit  of  data  for  the  first  quarter  of  FY200 1 .data  appear 
to  be  reported  correcily. 
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Light  Rail  Vehidej 

For  FY200 1 . quarterly  MDBF  data  for  each  of  the  four  quarters  were 
calculated  by  dividing  the  tool  mileage  for  the  quarter  by  the  total 
number  of  roadcalls  that  occurred  within  the  quarter.  For  FY2002, 
this  same  calculation  method  was  used  to  determine  the  MDBF  for 
die  first  and  fourth  quarters,  however  the  MDBF  for  the  second  and 
third  quarters  was  calculated  by  averaging  the  MDBF  for  the  months 
within  each  quarter. 

The  FY2002  third  quarter  MDBF  reported  for  the  entire  Light  Rail 
Division  does  not  match  source  data.  According  to  the  3rd  Quarter 
FY2002  Service  Standards  Report,  the  FY2002  third  quarter  MDBF 
for  the  Light  Rail  Division  was  2,237,  According  to  source  data,  the 
average  MDBF  for  the  Light  Rail  Division  should  be  2,657  for  the 
third  quarter  of  FY2002. 

Cable  Car 

Based  on  an  audit  of  data  for  the  fourth  quarter  of  FY2002,  data 
appear  to  be  reported  correctly. 

Prob/ems  or  Issues  Identified  During  the  Audit  Period 

Quarterly  MDBF  data  were  not  calculated  in  a consistent  manner 
for  light  rail  vehicles,  In  FY2002,  first  and  fourth  quarter  data  were 
calculated  by  dividing  the  toai  mileage  for  the  quarter  by  the  total 
number  of  roadcalls  that  occurred  within  the  quarter.  The  MDBF 
for  the  second  and  third  quarters  was  caiculated  by  averaging  the 
MDBF  for  the  months  within  each  quarter.  In  addition,  numerous 
calcuiation  errors  were  identified.  It  is  possible  that  audited  source 
data  were  adjusted  in  some  way  prior  to  reporting;  however,  no 
documenntion  of  those  adjustments  was  identified. 
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Chonges  in  Measurement  Since  the  Audit  Period  B 

Subsequent  to  the  audit  period.  Muni  has  made  no  changes  to  Tr 
the  definition  of  measurement,  method  of  measurement,  nor  the  res 
milestones.  wii 

ho 

Auditor  Recommendotions 


Quarterly  MDBF  should  be  calculated  in  a consistent  manner  - 
either  by  dividing  the  total  mileage  by  the  total  number  of  roadcalls. 
or  by  averaging  the  MDBF  for  the  three  months  within  a quarter 
A description  of  the  calculation  method  should  be  added  to  the 
section  describing  the  method  of  measurement 

K. 

T> 

au 

ot 
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B.  System  Performance 

Transit  system  pertormance  measures  assess  how  well  transit  agency 
resources  are  being  utilized,  how  efficiently  service  is  provided 
within  constraints,  how  effectiveiy  transportation  demand  is  met,and 
how  well  the  agency  is  administered.'  Proposition  E included  the 
following  service  standards  to  measure  Muni’s  system  performance; 

• lb.  Passengers  carried  by  mode. 

• 2b.  Fare  Revenue  generated  by  mode. 

• 3b.  Hours  and  miles  operated  by  mode. 

• 4b.  Expenses  incurred  by  mode, 

Key  Recommendations 

The  following  are  key  recommendations  based  on  the  findings  of  our 
audit  of  FY200I  and  FY2002  Service  Standards  data  and  on  trends 
observed  in  unaudited  FY2003  and  FY2004  Service  Standards  data; 

• For  Service  Standard  I b,  realistic  goals  for  ridership  should  be 
established  based  on  economic  conditions  as  well  as  available 
service. 

• Fare  revenue  is  an  important  management  tool  that  can  serve 
as  a surrogate  for  ridership  during  periods  of  fare  stability. 
Combined  with  other  Indicators,  It  is  possible  to  calculate 
average  fere  per  passenger,  a useful  management  tool  when 
looking  at  alternative  fere  structures.  While  this  Indicator 
should  continue  to  be  used,  the  current  practice  of  chang- 
ing the  benchmark  for  fere  revenue  annually  based  on  the 
prior  year’s  actual  experience  does  not  allow  for  reasonable 


'“A  GuJdebuokfof  O«r«lopl»t  iTnnst  Perfonrnnce-Meuurtinent  SyfXtrt"  TW  M.lOO). 
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comparison  across  time.  Rather  than  continuing  to  alter  the 
benchmark,  it  is  recommended  that  the  milestone  be  restated; 
"Fare  revenue  should  increase  by  1 .5%  per  year  compared  with 
prior  year  performance."  Changing  the  milestone  In  this  way 
will  allow  for  comparisons  over  time,  and  will  serve  as  a bet- 
ter indicator  of  the  need  to  adjust  feres  and  fere  struciurc. 

Mlles  and  hours  of  service  arc  a direct  reflection  on  the  avail- 
able budget.  It  Is  not  reasonable  to  expect  that  In  times  of 
budget  constraints  miles  and  hours  of  service  will  continue  lo 
increase,  or  even  remain  constant.  Miles  and  hours  of  service 
are  important  inputs  to  the  NationalTranslt  Database  (NTD) 
reporting,  and  will  be  calculated  annu.illy  whether  or  not  they 
continue  to  be  used  as  a performance  measure.  Consequently, 
the  CAC  and  MTA  Board  should  consider  eliminating  tills 
service  standard. 

The  goal  and  information  presented  for  Service  Standard  4b 
have  limited  value  to  the  public  or  to  Muni  itself.  In  fact.  Muni 
has  no  opportunity  to  exceed  its  budget  without  receiving  a 
supplemental  appropriation. as  a city  agency  cannot  operate  In 
deficit  Rather  than  comparing  expenses  to  budget  by  mode, 
which  Is  an  artificial  comparison,  it  Is  recommended  that  Muni 
track  fully  allocated  costs  per  hour  of  service  by  mode.  This 
standard  Industry  benchmark  would  allow  Mum  to  compare 
its  unit  costs  against  other  similar  properties  nationwide.  This 
information  1$  calculated  by  the  Finance  Department  regularly 
for  the  NTD  report  and  would  not  require  additional  ef- 
fort 
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lb.  Passengers  carried  by  mode. 


Gonk'^IP*'  '' 

TDincreasetheTiumberot;passeifne?bDardifig?by’niodBTB?j2%.  '1  ’ ' ' 

Purpose: 

Measuiement  of  the  ridership. 

Ooilnition  of  Moosurornont]^  > 

Annuaitne^eniBrn  oMhe  number  of  passengets.wljo^j^rd  the  Murii^jja^nug^vehicles.^^  passenflw 
^’countSeMhlimel^  a vehicle, even  t^ugh  they  may^be bn fhe;sarn%j^urneytfto.m;prigin to  . 

destination.  Data  will  be  reported  tVthVboar'd^on'a  quarterly  basis.  • 

Method  of  Moasuromont: 

Ride  checkers  are  utilized  to  count  the  passenger  boardings. 

Mllostonos: 

2%  increase  over  previous  year  ■ 

Frwn  Mum  S<nkt  JlondanJj  flcpoaFY  7001  end  FY700L 


Reported  Performance 

Ridcrshlp  dao  arc  reported  annually,  with  data  from  the  previous 
year  reported  for  the  current  fiscal  year  (l.c.,for  the  FY2002  Service 
Standards  Report,  ridership  for  FY200I  was  reported).  This  Is  due 
to  the  fact  that  final  ridcrshlp  calculations  cannot  be  completed  until 
several  months  after  the  conclusion  of  the  fiscal  year. 

For  both  FY2000  and  FY200I,  systemwide  ridership  increases 
exceeded  the  goal  of  1%  growth  from  the  previous  year.  However, 
this  growth  was  not  distributed  evenly  among  all  modes.  The  most 
significant  ridership  gains  were  for  light  rail  vehicles,  while  cable 


Table  B-l  Muni  Ridership,  FY I 999  - FY2003 


Division 

• ,^-.v 

FY1999 

1 

FY2000 

ji^rtreent 

Change 

(FV99TY00) 

FY2001 

’ Percent*^ 
Change 
(FY00-FY01) 

’■‘'^Percent’^  ; 

Change 

(FY99-FY011 

t " • 

’ FY2002 

r . - 

FY  2003  *' 

, Percent 
.Change  J 
f {fYOlQin 

K^Percent 

Change 

[■(PY0203I 

Moloi  Coach 

92,978.410 

96.394,514 

3.7S 

96,032,546 

-0.4S 

3.3% 

92.259.20} 

90.880.579 

■3.9% 

■1.5^^ 

Trollov  Coach 

70.275,162 

78.460,995 

0.2S 

80,868,518 

3,1  S 

3,3%; 

73,958.384 

74,398,945 

■8.5% 

, ...  0.6!: 

light  Rail  Vohiclo 

35.659.056 

41.610.040 

167S 

49,698,816 

19.4S 

39  4% 

44.976.474 

42.896.269 

■9.5% 

■4.6^ 

Coblo  Cot 

9,701,321 

9,206.298 

•5.1  S 

8,312,945 

-9.7S 

■14.3% 

7.257.683 

7.418.790 

■12.7% 

2.2'. 

TOTAL 

216,614,757 

225.671.847 

4.2S 

234,912.625 

4.1% 

8.4% 

218.461.742 

215.594.583 

■7.0% 

■1.V: 

From  AWm  Srmet  SMntoSi  Quani/Fr200l.rr}002.ri700J  «>iS  In  QuVttr  FY20O4 

Otu  m lolrci  n«i  bwi  iudie<l 


cars  lost  ridership  during  both  years.  Motor  coach  ridership  also 
fell  between  FY2000  and  FY2001.  Between  FYI999  and  FY2001 
ridership  on  all  modes,  except  for  cable  car,  increased  by  more  thar 
2%.  Ridership  decreased  by  7%  between  FY200I  and  FY2002.  anfl 
decreased  by  1.3%  between  FY2002  and  FY2003.  (This  decrease  is 
most  likely  due  to  the  overall  economic  downturn  that  occurrec 
during  this  time  period.)  Table  B-l  summarizes  ridership  and 
percent  change  in  ridership  for  fiscal  years  1999,  2000,  2001,  2002 
and  2003. 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Annual  ridership  was  esdmated  using  Muni’s  standard  nde  check 
protocols.  Baseline  ride  checks  arc  scheduled  to  occur  every  four 
I years.  Baseline  ride  checks  were  actually  conducted  about  every  five 
years  due  to  staff  turnover.  (In  other  words,  the  baseline  ridership 
for  a particular  line  would  be  reestablished  with  empirical  data  only 
every  five  years.)  Lines  that  had  not  been  checked  within  the  last 
four  years  were  given  priority  to  be  re-checked,  as  were  lines  that 
underwent  some  other  type  of  significant  change  (either  a route 
change,  or  significant  change  in  land  use  along  the  route).  Nearly  all 
lines  have  had  baseline  checks  since  1998 

Ride  checkers  carry  a handheld  elearonic  device  which  they  use  to 
count  boardings  and  alightings.  Typically,  one  checker  is  assigned  to  a 
vehicle.  Counts  are  electronically  uploaded  from  the  handheld  devices 
onto  computers.  Specialized  software  is  then  used  to  calculate  loads 
and  check  that  boardings  and  alightings  balance  out.  Checkers  also 
count  the  number  of  trips  that  occur  within  a given  time  pcriod. 
(This  information  is  entered  into  the  computer  software  as  well.) 
The  daily  boardings  during  a given  time  period  are  calculated  by 
multiplying  the  average  boardings  per  trip  and  the  actual  number  of 
trips  that  occurred  within  a given  time  period. 

Raw  data  are  edited  before  baselines  are  calculated  by  the  RUCUS 
software  program  (developed  specifically  for  Muni).  Baseline  data 
are  used  to  develop  a conversion  factor  allowing  ridership  at  the 
maximum  load  point  on  a line  to  be  converted  to  a ridership  estimate 
for  an  entire  line.  This  methodology  has  been  approved  by  the 
Federal  Transit  Administration  (FTA).  During  the  periods  between 
baseline  checks,  checks  are  only  made  at  maximum  load  points. 
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During  the  reporting  period,  each  line  was  checked  at  Its  maximum 
load  point  (MLP)  for  two  days  from  5 AM  - 7 PM.  Some  lines  were 
checked  only  at  one  MLP,  while  others  were  checked  at  several  MLPs 
(different  points  were  checked  for  different  time  periods  for  some 
lines).  Because  the  software  system  only  recognizes  published  time 
points,  the  MLP  was  chosen  to  correspond  to  a time  point  as  close 
as  possible  to  the  true  MLP.  The  two-day  check  data  was  averaged, 
and  the  average  load  and  number  of  trips  were  then  calculated, 

The  computer  software  automatically  calculated  the  number  of  trips 
that  needed  to  be  checked  for  statistical  significance.  Extra  checks 
were  often  made  in  order  to  ensure  a valid  count. 

Annual  ridership  calculations  for  individual  lines  arc  subject  to  a 
greater  error  than  the  annual  ridership  calculation  for  the  entire 
system.  For  the  annual  system-wide  ridership  estimate,  there  Is  a 
95%  chance  that  the  estimate  Is  within  ±10%  of  the  actual  annual 
ridership  based  on  this  approved  sampling  methodology 

Venficotion  of  Reported  Performance  During  the 
Audit  Period 

Dau  reported  In  the  Service  Standards  Reports  were  compared  to 
that  reported  in  the  Federal  Transit  Administration’s  (FTA)  National 
Transit  Database  (NTD).  (All  data  reported  to  the  NTD  arc  audited 
regularly  by  the  FTA.)  All  data  were  found  to  be  consistent. 
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Problems  or  Issues  Identified  During  the  Audit  Period 
No  significant  problems  or  Issues  were  identified, 

Chonges  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  adjusted  the  milestones  to 
compare  passenger  boardings  to  the  previous  year.  The  milestone 
for  FY2003  and  FY2004  was: 

* 224  Million  Passenger  Boardings 
Auditor  Recommendat/ons 

The  definition  of  measurement  should  be  amended  to  remove  the 
statcment.‘‘Data  wilt  be  reported  to  the  board  on  a quarterly  basis." 
Realistic  goals  for  ridcrshlp  should  be  established  based  on  economic 
conditions  as  well  as  available  service. 
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2b.  Fare  revenue  generated  by  mode. 


Goal: 

To  increase:fare  revenue  overall  by  $1.6M  compared  with  FY2000 

Purpose: 

Measure  tare  revenue  by  mode,  as  well  as  by  General  Fast  Pass  Sales 

Definition  of  Measurement:  : 

Fare  revenue  ci)ilection  on  board  revenue  vehicles,  as  well  as  sales  of  individual  tickets  and  passes  of  all  types, 
includirio  shares  of  regional  passes  such  as  BART  and  spectal  event  passes. 

Method  of  Measurement: 

Cash  fares  are  collected  electronically  on  board  all  revenue  vehicles  {with  the  exception  of  Cable  Car),  utilizing  the 
Cubic  Farebox  system.  In  Cable  Car,  a manual  fare  collection  system  along  with  sate  of  special  passes  is  utilized. 
POP  stations  sell  tickets  at  the  platform. 

Milestones: 

FY2001  - fare.revenue  increase  of  $1.96  M exceeded  standard  FY2002-  fore  revenue  increase  of  $2.08  M 
exceeded  standard 

From  Muni  Jeroce  Unndonj!  Ft  2001  0n4  FY200i. 


Reported  Performance 

The  established  goal  of  increasing  fare  revenue  by  $1.6  M over 
FY2000  fare  revenue  was  accomplished  for  both  FY200 1 and  FY2002 
Mum  was  unable  to  achieve  the  target  revenue  of  $100  M in  FY2003. 
Table  B-2  summarizes  data  on  fere  revenue  for  FY2000  - FY2003. 

Table  B-2  Annual  Cash  Fare  Summary.  FY2000  - FY2003_ 


Mode  Annual  Cash  Fara  Sutiwnarv 


. FY2000.  ' 

1 

• :.fy200i_; 

j '-Improvement 
:FY2000 to FY2001 

FY2D02 

Improvement 
FY200QIO  FY2002 

' F2003 

tmprovemnnl 
FY2000IO  FY2003 

Motor  Coach 

$15,067,705 

$14,504,449 

$563,256 

1 $15,949,862 

$002,157 

$14,040.0$? 

$1.027617' 

toliey  Coach 

$13,881,967 

-$13,576,560 

-$305,407 

$13,431,373 

$450,594 

: 12.249.773 

$1,632,188 

light  Rail 

$8,542,470 

$9,555,224 

$1,012,754 

$0,254,448 

$288,022 

f 7.910.161 

$■632,309 

^able  Car 

: $12,835,482 

$12,385,518 

•$449,964 

$12,385,518 

-$449,364 

$11,008,050 

$1,827,432 

Fast  Pass  Sales 

$45,427,773 

$47,252,188 

$1,024  415 

$47,537,277 

$2,109,604 

$44  81 7. 894  \ 

$609879 

bihef  Medii"  r 

$5,554,630' 

$5,982,166 

$427,536 

$5,731,066 

$176,436 

$6,255,035 

$700,405 

Paratiansii 

$764,616 

$782,230 

$17614 

$865,741 

$101,125 

$1,071,099 

$306,483 

Charter  Servire»t 

■ - • 

• 1 

* « * ♦ 

$15,609 

$15,609 

Total 

$102,074,643 

$104,038,335 

$1,963,692 

$104,155,285 

$2,080,642 

$97367.714 

4 400.445 

From  Mur*  Semce  Siondani  fiepori4tft  Quarter  flT200(.FY7<Xlirn003  end  In  Quarter  Ff200* 
Dan  In  talics  not  b««ft  audited. 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Fare  revenue  data  were  collected  by  the  Revenue  Department 
and  reported  monthly  to  the  Finance  Department  The  Revenue 
Department  provided  information  on  pass  sales  and  sales  of  other 
media  as  well  as  revenue  collected  on  board  vehicles.  The  Revenue 
Department  provided  revenue  by  division,  which  enabled  Finance  to 
allocate  cash  revenue  by  mode. 

The  auditor  was  unable  to  verify  that  the  reports  provided  to  Finance 
accurately  reflected  the  amount  of  revenue  collected,  however  a 
review  of  the  controls  on  revenue  data  showed  a highly  controlled 
and  automated  environment  that  meets  FTA  audit  requirements. 


VerlfJcotion  of  Reported  Performance  During  the 
Audit  Period 

Based  on  an  audit  of  the  monthly  revenue  reports  and  the  calculation; 
completed  by  the  Finance  Department,  all  revenue  data  appeared  w 
be  reported  correctly. 

Problems  or  fssues  Identified  During  the  Audit  Period 

Performance  In  FY200I  and  FY2002  were  both  compared  with 
FY2000  and  were  both  found  to  exceed  the  standard.  Assuming 
each  fiscal  year  was  intended  to  be  compared  to  the  prior  year': 
performance,  and  not  to  the  FY2000  benchmark,  the  following  result 
is  calculated: 
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|ln  this  case,  the  milestone  of  a $1.6  M increase  in  fare  revenue  was 
exceeded  in  FY200I.  In  FT2002,  the  target  of  an  additional  $16 
M increase  in  fere  revenue  was  substantially  missed,  with  only  a 
$1 16,950  increase  in  fere  revenue  between  FY200I  to  Pl'2002. 

Chonges  in  Meosurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  adjusted  the  milestones  to  a 
fixed  milestone,  rather  than  comparing  fere  revenues  to  the  previous 
year.  The  FT2003  and  FT2004  milestones  were: 

• Achieve  $100  M m fere  revenue  in  Pi'2003 

• Achieve  $1 17  M in  fere  revenue  in  FY2004 

Action  Plan 

Fare  revenue  is  an  important  management  tool,  and  can  be  a surrogate 
for  ndership  during  periods  of  fere  stability.  Combined  with  other 
Indicators,  it  is  possible  to  calculate  average  fere  per  passenger,  a 
useful  management  tool  when  looking  at  alternative  fare  structures. 

While  this  indicator  should  continue  to  be  used, the  current  practice  of 
changing  the  benchmark  for  fere  revenue  annually  based  on  the  prior 
year’s  actual  experience  does  not  allow  for  reasonable  comparison 
across  time.  Rather  than  continuing  to  alter  the  benchmark,  it  Is 
recommended  that  the  milestone  be  restated;  "Fare  revenue  should 
Increase  by  1 .5%  per  year  compared  with  prior  year  performance. 
Changing  the  milestone  in  this  way  will  allow  for  comparisons  over 
time,  and  will  serve  as  a better  indicator  of  the  need  to  adjust  feres 
and  fere  structure. 
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3b.  Hours  and  miles  operated  by  mode. 


Goal;  ~ 

iTo'Increase  hours  and.miles  operated  by  modeJiyt1f2%  ''  ’ ' 

Purpose: 

Measure  the  distribution  of  miles  and  hours  traveled  by  mode. 

Definition  of  Measuromont: 

The  miles  and  hours  that  vehicles  travel  while  in  revenue  service.  Vehicle  reveiiue  miles  and  rei/enue  hours 
exclude  travel  to  and  from  storage  facilities,  training  operators  prior'tbTwenUe’-service;  rdad'tests’a'nd-deadheadl^ 
travel,  as vvell as charterservIcB.''^'"'  ’ ' . ..  ! 

Method  ol  Moasuromeni: 

Reported  from  schedules,  excluding  any  service  interruptions  or  special  additional  service. 

Milestones; 

v-ii- ' . 

From  Muni  Service  SworfonSl  fieporl  fY  3001  ond  FTJW2. 


Reported  Performance 

Data  for  hours  and  miles  are  reported  annually,  with  data  from 
the  previous  year  reported  for  the  current  fiscal  year  (I.e.,  for  the 
FY2002  Service  Standards  Report,  hours  and  miles  for  FY200I 
wore  reported).  This  is  due  to  the  fact  that  final  mileage  and  hour 
calculations  cannot  be  completed  until  several  months  after  the 
conclusion  of  the  fiscal  year. 


For  both  FY2000  and  FY200 1 .systemwide  mileage  and  hour  increases 
exceeded  the  goal  of  1.2%  growth  from  the  previous  year.  However, 
this  growth  was  not  distributed  evenly  among  all  modes.  The  most 
significant  gains  were  for  light  rail.  This  trend  continued  in  FY2002 
and  FY2003.  Table  8-3  summarizes  the  hours  and  miles  and  percent 
change  in  hours  and  miles  for  fiscal  years  1 999, 2000, 200 1 . 2002  and 
2003. 


Table  B-3  Revenue  Hours  and  Miles,  FY  1999  - FY2003 


Oiu  In  iQjIa  Stv«  not  b*«n  iud>(ad 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Annual  hours  and  miles  were  calculated  from  schedules.  The  number 
of  miles  was  derived  from  the  number  of  hours  and  the  average 
travel  speed  of  vehicles.  Missed  service  (from  the  Missed  Service 
Report)  was  subtracted  from  the  number  of  scheduled  hours  and 
miles  to  determine  the  actual  number  of  miles  and  hours. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Data  reported  in  the  Service  Standards  Reports  were  compared  to 
that  reported  in  the  Federal  Transit  Administration’s  (FTA)  National 
Transit  Database  (NTD).  (All  data  reported  to  the  NTD  are  audited 
regularly  by  the  FTA.)  All  data  were  found  to  be  consistent  with  the 
exception  of  the  number  of  miles  and  hours  reported  for  motorcoach 
in  FY200 1 . The  number  of  hours  and  miles  reported  in  the  Service 
Standards  Report  was  i. 405.682  hours  and  I2.4ZZ.997  miles.  The 
number  of  hours  and  miles  reported  to  NTD  was  1.405.628  hours 
and  1 2.44Z.997  miles.  (It  seemed  as  though  numbers  were  mistakenly 
inverted  in  the  Service  Standards  Report) 

Problems  or  Issues  Identified  During  the  Audit  Period 

No  problems  or  issues  were  identified, however  much  of  the  hours  and 
miles  data  must  be  tabulated  manually,  which  is  quite  time  intensive. 
Once  the  new  Trapeze  scheduling  software  is  implemented,  most  of 
the  calculations  can  be  performed  automatically,  which  should  save  a 
significant  amount  of  staff  time. 
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Chonges  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  adjusted  the  milestones  to  a 
fixed  milestone,  rather  than  comparing  fare  revenues  to  the  previous 
year.  The  FY2003  and  FY2004  milestones  wore: 

• Revenue  Hours  Goal;  3.6  M hours  In  FY2003  and  3.5  M hours 
in  FY2004 

• Revenue  Miles  Goal:  25  M miles  in  FY2003  and  FY2004 
Action  Plan 

While  there  were  no  problems  identified  with  the  collection  or 
reporting  of  data,  it  should  be  noted  that  miles  and  hours  of  service 
are  a direct  reflection  of  the  available  budget.  It  Is  not  reasonable  to 
expect  that  in  times  of  budget  constraints  miles  and  hours  of  service 
will  continue  to  increase,  or  even  remain  constant 

Miles  and  hours  of  service  are  important  inputs  to  tlic  NationalTransit 
Database  reporting,  and  will  be  calculated  annually  whether  or  not 
they  continue  to  be  used  as  a performance  measure.  Consequently, 
the  CAC  and  MTA  Board  should  consider  eliminating  this  service 
standard. 
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Expenses  incurred  by  mode. 


Goa):  ^ 

■Fxbenses  bv  MajJajn  each  Fiscal')!rarTd“femain  within  budgp'd^mouni. 

Purpose: 

Measure  the  cost  ot  ptoducino  revenue  sendee  by  mode  as  well  as  budget  adherence. 

DofInUIonof  * 

MoQSuramont: 

Judgments  and  Claims,  Intctfund  transfers,  use  of  Muni  fast.passes^n|ART, 

Benefits  fonetirees,  long  term  leaves,  and  services  of  the  City  attorney  ■ ' • z 

Method  ot  Monsuroment; 

Budget  expenses  are  Itacked  by  line  item  and  section  but  are  rolled  up  into  mode 
annually.  Data  reported  on  an  annual  basis. 

Mllostonos; 

FYZuOl^^penses  are  wiinin  ouagei‘r>i^uu<«B^pBitMaHtsiwiiiim  uuuuoi-.v-j. 

From  Wun(  Strykt  Sundordt  /IcpeflfY  2001  omJflQOOZ 


Reported  Performance 

The  annual  Service  Standards  Report  simply  reports  whether  costs 
are  within  budget  or  not  In  both  audit  years,  cosu  were  reported 
within  budget. 

Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Muni's  expenses  were  allocated  to  expense  codes  In  four  major 
categories; 

• Vehicle  (including  driver  wages); 

• Vehicle  niaintcnancc; 

• Administration;  and 

• Non*Vchlclc  Maintenance  (l.e„  Facilities). 


must  be  allocated  to  a mode  using  a system  approved  by  FTA  for 
the  National  Transit  Database.  The  methodology,  approved  in  1987 
allows  Muni  to  allocate  non-mode  specific  costs  to  each  mode 
through  a formula  based  on  the  revenue  miles  by  mode. 

The  measurement  reported  in  the  Service  Standard  Reports  was 
consistent  with  the  NTD  information  and  is  derived  from  NTD 
reports.  The  actual  measurement  is  slightly  different  from  the 
method  described  in  the  table  above.  In  fact,  a correct  statement  of 
measurement  would  be; 

Expenses  collected  ond  allocated  by  mode,  exclude  Worker's 
Comp,  use  of  Muni  fost  posses  on  BART!  benefrts  for  retirees 
not  poid  by  Muni,  ond  non<ompensoted  long  term  leaves. 

This  definition  Is  consistent  with  the  NTD  report,  and  therefore 
more  accurately  reflects  what  is  currently  being  reported. 


Within  each  category,  the  mode  can  be  coded,  to  allow  costs  to 
be  tracked  by  mode.  While  many  costs  track  directly  to  mode  in 
a straightforward  manner,  costs,  such  as  most  administrative  costs. 
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‘ Verification  of  Reported  Performance  During  the 

l'  Audit  Period 
1 

i The  audit  team  confined  its  investigation  to  the  reporting  done 
I by  the  Finance  Department.  The  auditor  was  therefore  unable  to 
comment  on  the  accuracy  of  costs  reported  to  each  budget  line  item. 
However,  the  audit  examined  the  process  and  controls  involved  in 
preparing  the  expenditure  reports  that  are  used  in  NTD  and  Service 
Standard  reporting.  All  appropriate  controls  appeared  to  be  in  place 
I to  track  costs  by  mode  in  an  acceptable  manner. 

Problems  or  Issues  Identified  During  the  Audit  Period 

The  Service  Standard  Report  simply  states  whether  or  not  Muni  was 
“in  budget"  by  mode.  It  does  not  provide  modal  comparisons,  nor 
does  it  provide  line  item  expenses.  Even  the  information  presented 
Is  a bit  misleading,  since  some  elements  of  the  Muni  budget  are  not 
mode  specific,  although  they  are  ultimately  allocated  by  mode  based 
on  the  NTD  formulas. 
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Chonges  in  Measurement  Since  the  Audit  Period 

Beginning  in  FY2003,  this  Service  Standard  was  retitlod  "Budget 
versus  Actual." 

Action  Plan 

This  goal  and  the  information  presented  have  limited  value  to  the 
public  or  to  Mum  itself.  In  fact,  Muni  has  no  opportunity  to  exceed 
its  budget  without  receiving  a supplemental  appropriation,  as  a City 
agency  cannot  operate  in  deficit.  Rather  than  comparing  expenses  to 
budget  by  mode,  which  is  an  artificial  comparison.it  Is  recommended 
that  Mum  track  fully  allocated  costs  per  hour  of  service  by  mode.  This 
standard  industry  benchmark  would  allow  Muni  to  compare  its  unit 
costs  against  other  similar  properties  nationwide.  This  information 
is  calculated  by  the  Finance  Department  regularly  for  the  NTD 
report,  and  would  not  require  additional  effort.  A recommended 
performance  measurement  system  would  include  the  following: 


Goa]?""  ' : 

/Fully  ellocated  uhit-cost'per  hour  for  each  mode  will  not  increase  by  more  than  the  Bay  Area  CPI  annually. 

Purpose: 

Measure  the  cost  of  producinr)  revenue  service  by  mode  as  well  as  the  ability  to  corrtrol  unit  costs. 

Definition  of  | 

Measurement: 

•1 

j'Expehsis  collected  and  allocated  by  mode  will  be  based  on  the  NTO  methodology,  (exclude  Worker's  Comp,  use  of  Muni 
ifast  passes  on  BART'benefits  for  retirees  not  paid  by  Muni,  and  noH'Compensated  long  term  leaves]  divided  by  the 
'number  of  hours  operated  by  mode  to  calculate  unit  costs  per  hour. 

Method  of  Measurement: 

Expenses  by  mode  are  calculated  using  an  NTO  approved  methodology  on  an  annual  basis.  Annual  hours  by  mode  aro 
calculated  ustng  an  NTO  approved  methodology  on  an  annual  basis. 

Milestones':'  " J i 

[Costsi^i^hour  of  service  for  each  mode  shall  not  exceed  the  Bay.  Area  CPI  in  each  year. 

t 


I 
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Z.  Staffing  Performance 

"he  Proposition  E Service  Standards  for  staffing  performance  focus 
in  staffing  levels  and  the  ability  to  retain  new  employees.  They 
iclude; 

• Ic.  Net  vacancies  by  position  (vacancies  remaining  once 
promotions  and  new  hires  have  been  deducted  from  retirees 
or  resignations)  for  each  division. 

• 2c.  Attrition  rates  for  new  employees,  by  division  and  level 

'ey  Recommendations 

he  following  are  key  recommendations  based  on  the  findings  of  our 
jdit  of  FY200I  and  FY2002  Service  Standards  data  and  on  trends 
jserved  in  unaudited  Pf'2003  and  Pf2004  Service  Standards  data: 

• For  Service  Standard  I c,  the  great  majority  of  vacancies  cannot 
be  filled  due  to  budgetary  policies.  In  order  to  ensure  that 
service  provision  does  not  suffer.  Muni  has  focused  on  making 
sure  that  Maintenance  and  Operator  positions  are  filled  Con- 
sequently. the  auditor  recommends  that  this  Service  Standard 
be  limited  to  Transit  Operators  and  Maintenance  personnel, 
or  that  the  goals  for  other  divisions  (exclusive  ofTransit  Op- 
erators and  Maintenance  personnel)  be  revised  so  that  they 
can  be  reasonably  met  given  budgetary  constraints. 
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• In  order  to  track  the  attrition  rate  for  new  hires  In  Service 
Standard  2c,  attrition  rates  should  either  be  reported  at  the 
end  of  new  employees’  probationary  periods  or  witiiin  one 
year  after  hire.  To  calculate  attrition,  only  those  employees 
who  separate  after  six  months  or  one  year  of  service  should 
be  counted-  Furthermore,  there  is  no  information  provided  in 
this  measure  about  why  employees  are  voluntarily  separating 
from  Mum  service.  If  it  is  possible  to  obtain  this  information 
from  the  Position  Control  System  database,  then  it  should 
also  be  summarized  and  included  in  this  measurement. 


Ps9«C-l 
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Ic. 


Net  vacancies  by  posit, on  (vacancies  ren.o,ning  once  promotions  ond  new  hires  hove  been 


Gonl:  ^ 
PuiPOSU. 

— — n.i.^^aBgSrr3Hw^.  mm 

'No  nroatcr  than  5%  vacancy  rate.  

Mtinonru  IbubI  nf  itifi  deodUmtni  hi  liilihu).  nil 

biBWiiiiBIIIIIIIIIHWiWWIWWI^^  rsnatnSifl^rinBfRrfnDsitiDnshvWiFiSippm  " , 

'UOiinition  m MOBSuromoni: 

Ti,J!!rJrv^nrfTll  be  the  basis  of  the  data  reponed  to  the  Board  on  a quarterly  basis. 

Mulhou  Of  Monsuromoni. 

tiomMonI  Jefvtte  Jtondofdi  Rtperj.fYJMt  enafY200i, 


Reported  Performance 

The  number  of  budgeted  positions  and  the  number  and  percentage 
of  vacancies  at  the  beginning  and  end  of  each  quarter  were  reported. 
Table  C*l  shows  tl^o  percentage  of  scheduled  positions  that  were 
vacant  at  the  end  of  the  fourth  quarter  for  fiscal  years  2001,  2002 
and  2003  as  well  as  at  the  end  of  the  second  quarter  of  FY2004. 
Overall.  Muni  met  Its  goal  of  having  a vacancy  rate  of  less  than  5%  for 
all  four  years.  The  vacancy  rate  for  transit  operators  was  the  lowest, 
while  that  for  Finance  and  Administration  and  GM  Programs/Safety 
and  Training  was  highest 


C- 1 Vacancy  Rate,  FY200 1 - FY2004 


FY200) 

FY2002 

FY2003 

FY2 

1.  ■ .t 

004  1 

' vi.'*^.p<aaa 

■i^Hn 

^Gudootod 

^Posltlonr' 

t'Vacancias; 

QtEndol  j 
Hth  Quarter 

uBudgoted 

^Positions 

''tfacancifis! 
at  End  of  ' 
4th  Quaiterj 

Budgeted  , 
impositions 

1 Vacanciesfl 
at  End  of  1 

rt; — n 
1 Budgoled'l 
IpPasitibns  ' 

(VCc^ncLBSati 
ilndohnd  ‘ 
!.^*Quarter  ^ 

Transit  OporMors 

2.176 

1.9% 

2,176 

1.1% 

U63 

0% 

1 1163 

0% 

MainionancolOps  Admlnistiotivo 

1.540 

6.6% 

1.639 

5.1% 

1.743 

.6.7% 

1,685 

. 8.0% 

Finance  and  Ailminisliation 

244 

10.7% 

222 

6.6% 

240 

12.3% 

242\ 

12.5% 

GM  PfooromsJSalolv  & Troinmi)' 

■‘263 

8.3% 

^ 161 

17.4% 

m 

18.3% 

203 

22.8% 

Totals 

4,213 

A5% 

4.199 

3.7% 

4.310 

4.2% 

4.293 

4.7% 
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Veri/icfltion  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

The  Annual  Salary  Ordinance  was  used  to  determine  the  number  of 
budgeted  positions  for  each  fiscal  year. “Vacancies”  were  any  type  of 
permanent  separation,  including  releases,  dismissals,  retirements  and 
transfers.  Leaves  were  not  counted  as  vacancies.  Vacant  positions 
were  Dbulated  using  the  Position  Control  System  database.  The 
database  was  queried  for  all  vacancies,  since  it  cannot  create  separate 
vacancy  queries  for  each  department.  (This  was  done  manually.) 
Staff  checked  to  ensure  that  vacancies  tabulated  for  the  end  of  the 
previous  quarter  matched  the  number  of  vacancies  reported  for  the 
beginning  of  the  subsequent  quarter. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Based  on  an  audit  of  data  for  the  third  quarter  of  FY2002,  vacancy 
rates  appear  to  have  been  calculated  and  reported  correctly. 

Prob/ems  or  issues  Identified  During  the  Audit  Period 

If  the  Position  Control  System  database  could  be  queried  to  report 
the  number  of  vacancies  by  department,  it  would  save  a significant 
amount  of  staff  time. 
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Changes  in  Meosurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  has  made  no  changes  to 
the  definition  of  measurement,  method  of  measurement,  nor  the 
milestone. 

Auditor  Recommendations 

The  great  majority  of  vacancies  cannot  be  filled  due  to  budgetary 
policies.  In  order  to  ensure  that  service  provision  does  not  suffer, 
Muni  has  focused  on  making  sure  that  Maintenance  and  Operator 
positions  arc  filled.  Consequently,  the  auditor  recommends  that  this 
Service  Standard  be  limited  to  Transit  Operators  and  Maintenance 
personnel,  or  that  the  goals  for  other  divisions  (exclusive  of  Transit 
Operators  and  Maintenance  personnel)  be  revised  so  that  they  can 
be  reasonably  met  given  budgetary  constraints. 


P*7»03 
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2c.  Attrition  rotes  for  new  emJ>(oyees,  by  division  ond  level. 


Coil:  ■ 

^ — r- - V'  

'Nopreatep.thanM'S;..  - ••  

Purposo: 

Measutemeni  of  effectiveness  of  tectuilment  and  employee  satisfaction  by  the  rate  of  voluntary 
seoarations  for  new  employees. 

Dolinition  of  Moasuromunl: 

• - 

Number  <»(  and  claa|^|iD  aie  i^sed^Ufmg.pToba^ry,  periQ|l;or.yyho 

arovoiuVlariif'^epMflimirp^^ 

as  well  as  attrition  one  year  after  hire.  Data  will  he  leporteoto  thsiboard  on  a qua^e^|^,(basib«. . 

Method  of  Moosuroment: 

Vacancy  Reoort  will  provide  data  (or  quarterly  reporting. 

Mllostoiios: 

From  Muni  Service  Sionderdi  Heperi,rf  20Ot  orul  FY300J. 


Reported  Performance 

Each  quarter,  the  number  of  new  hires,  releases,  and  voluntary 
separations  that  occur  within  each  division  were  reported.  The 
attrition  rate  was  calculated  by  dividing  the  total  number  of  releases 
and  volunory  separations  that  occur  within  a quarter  by  the  total 
number  of  new  hires  within  that  quarter.  During  the  two  fiscal 

Table  C-2  Attrition  Rate,  FY200 1 - FY2004 


4th  Quirttr  FY2QQ1  4th  Quarter  FY2002  4th  Quarter  FY2003  2nd  Quarter  FY2004 
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years  audited.  Muni  never  achieved  an  attrition  rate  lower  than  1 1 % 
(which  occurred  during  the  first  quarter  of  FY200 1 ).  Attrition  rates 
remained  high  in  FY2003  and  FY2004,  as  shown  in  Table  C-2. 
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The  attrition  rates  reported  in  the  Service  Standards  Reports  do 
not  accurately  represent  actual  attrition  rates  for  new  employees, 
however.  The  rate  was  based  on  the  number  of  people  who  were 
hired  or  who  separated  during  a quarter,  however  the  employees 
who  separated  during  a quarter  were  not  necessarily  the  same  as 
those  who  were  hired  during  that  quarter.  Furthermore,  voluntary 
separations  included  anyone  who  voluntarily  separated,  regardless 
of  their  length  of  employment.  This  issue  is  discussed  further  in  the 
subsequent  sections. 

Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

The  Position  Control  System  database  was  used  to  determine  the 
number  of  new  hires,  releases  and  voluntary  separations  within  each 
quarter.  Releases  were  counted  as  those  staff  who  were  released 
for  disciplinary  or  non-disciplinary  reasons  during  their  probationary 
period  (typically  the  first  six  months  of  employment).  Voluntary 
separations  included  staff  who  either  quit  work  or  transferred.  It 
did  not  include  retirements.  The  number  of  voluntary  separations 
reported  in  a given  quarter  were  for  anyone  who  left  during  that 
quarter,  regardless  of  the  length  of  his  or  her  employment. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Based  on  an  audit  of  data  for  the  third  quarter  of  FY200 1 . attrition 
rates  were  not  calculated  correctly.  Voluntary  separations  included 
all  employees  who  separated  from  Muni  during  a particular  quarter, 
regardless  of  their  length  of  employment. 
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Problems  or  Issues  Identified  During  the  Audit  Period 

Attrition  rates  were  not  reported  for  both  the  probationary  period 
and  for  one  year  after  hire,  per  the  stated  definition  of  measurement. 
Either  data  should  be  tabulated  differently,  or  the  stated  definition 
of  measurement  should  be  amended  as  follows;  “Will  break  down 
attrition  within  probationary  period." 

The  reporting  methods  used  during  FY200I  and  FY2002  do  not 
provide  information  on  the  attrition  rates  for  new  employees.  The 
employees  who  were  hired  within  a particular  quarter  were  not 
necessarily  the  same  employees  who  separated  within  that  same 
quarter 

Changes  fn  /Vleasuremenf  Since  the  Audit  Period 

Beginning  in  FY2004.the  attrition  rate  for  each  quarter  was  calculated 
by  dividing  the  total  number  of  releases  and  voluntary  separations 
that  occured  within  the  quarter  by  the  toul  number  of  new  hires 
made  in  the  last  12  months. 

Auditor  Recommendotions 

In  order  to  track  the  attrition  rate  for  new  hires,  attrition  rates 
should  either  be  reported  at  the  end  of  new  employees’  probationary 
periods  or  within  one  year  after  hire.  To  calculate  aitniion.only  those 
employees  who  separate  after  six  months  or  one  year  of  service 
should  be  counted.  Furthermore,  there  is  no  information  provided 
in  this  measure  about  why  employees  arc  voluntarily  separating  from 
Muni  service.  If  it  is  possible  to  obtain  this  information  from  the 
Position  Control  System  database. then  it  should  also  be  summarized 
and  included  in  this  measurement 


P»jtC4 
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D.  Customer  Service 

Customer  service  addresses  passenger  safety  on  board  vehicles  and 
at  stops;  the  ease  with  which  passengers  can  navigate  and  use  the 
transit  system  to  get  where  they  need  to  go  in  a timeiy  and  efficient 
manner:  the  availability  of  information  about  schedules,  delays  or 
other  service  interruptions;  and  the  responsiveness  of  transit  agency 
staff  when  customer  problems  or  complaints  arise.  Proposition 
E includes  the  following  Service  Standards  aimed  at  measuring 
customer  service: 

• Id.  Development  of  an  annual  marketing  plan  identifying 
specific  programs  and  projects  that  will  promote  increased 
patronage. 

• 2d-  Publication  and  distribution  to  the  public  of  schedules  for 
all  trips  taken  by  all  vehicles  that  shall  consists  of  specific  ar- 
rival times  at  terminals  and  established  intermediate  points. 

• 3d.  Operator  conduct  complaints. 

• 4d.  Annual  passenger  surveys  and  follow-up  by  manage- 
ment, 

• 5d.  Improvements  in  public  information  regarding  vehicle 
delays  during  operations  as  well  as  any  general  user  infor- 
mation regarding  system  modifications,  route  changes,  and 
schedules. 

• 6d.  Efforts  to  improve  driver  training,  technical  as  well  as  ac- 
cident follow-up. 

• 7d.  Number  of  crime  incidents  on  Municipal  Railway  vehicles 
or  in  Municipal  Railway  facilities. 
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Key  Recommendations 

The  following  are  key  recommendations  based  on  the  findings  of  our 
audit  of  FY2001  and  FY2002  Service  Standards  data  and  on  trends 
observed  in  unaudited  FY2003  and  P^2004  Service  Standards  data: 

• For  Service  Standard  I d, marketing  priorities  should  be  m.ade 
within  budget  constraints  and  with  the  goal  of  increasing 
patronage:  data  from  rldcrship  surveys  should  be  used  to 
evaluate  the  success  of  marketing  efforts. 

• For  Service  Standard  2d,  the  current  process  of  schedule 
preparation  is  cumbersome,  time  consuming,  and  relics  on 
the  background  and  knowledge  of  one  key  staff  person.  The 
capabilities  of  the  new  Trapeze  software  should  be  fully  utilized 
in  order  to  simplify  and  streamline  the  process  of  schedule 
preparation. 

• For  Service  Standard  3d.  operator  conduct  complaints  need 
to  be  more  clearly  measured  and  reported,  realistic  goals  for 
reducing  the  number  of  operator  conduct  complaints  should 
be  established,  and  the  efficiency  and  accuracy  of  complaint 
information  uptake  should  be  improved. 

• In  order  to  meet  the  accident  reduction  goals  csobllshcd  in 
Service  Standard  6d.  it  will  be  necessary  to  develop  a compre- 
hensive approach  and  coordinated  effort  to  reduce  accidents 
across  divisions  and  departments.  Driver  training  was  found 
to  be  sufficient.  Therefore,  the  focus  of  accident  reduction 
efforts  should  be  enforcement  of  safety  rules  and  procedures 
and  accident  prevention  through  thorough  review  of  all  ac- 
cidents. Additionally,  it  would  be  useful  to  report  types  of 
accidents  according  to  those  that  occur  during  revenue  and 
non-revenue  hours,  and  those  that  were  determined  to  be 
preventable  and  non-prcvcntable. 
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• To  be  truly  useful.  Service  Standard  5d  should  be  divided  into 
two  goals  to  distinguish  between  routine  informatjon  provided 
In  real  time  and  longer-term  Information  provided  as  part  of 
service  or  schedule  change  and/or  construction  projects.  To 
further  differentiate  between  immediate  information  provided 
to  passengers  on  the  system  during  delays  or  other  unusual 
events,  and  longer-term  Information  provided  to  address  con- 
struction or  other  longer-term  events,  the  following  systems 
of  measurement  arc  recommended: 


Goal; 

ImorovB  Real-TimB^ssaiTner  InfotmaTion  tBgaTdinq  se™cafflalaysJJ*4*iy(#IW*iil^^  '■  1 

Purposo: 

Provide  the  public  with  information  regarding  service  delays  that  will  enable  them  to  make  choices  about  ttieir  trip. 

Ooflnitioh  of  Moosuromont:  ' 

Improve  passenger  information  by  communication  of  service  ptoblemsand  other  information  to  each’Vehtcle,  thestajimt 
platforms,  the  Telephone  information  Center,  media,  the  Service  Hotline,  and.Service.Qijality  Teams. 

Operators  will  make  passenger'announcements  in  all  cases  where  service  dela^exceed^  rhinut«  in  the  sii'bwa^'and  10 
minutas  on  surface  lines.  - 

Central  Control  will  make  both  visual  and  verbal  announcemems  in.alLsUi;ipns  where  service  delays  exceed  ,3  ptinutes  bn-all 
LRV  lines. 

Service  Quality  Teams  will  be  assigned  to  all  unscheduled  service  disruptions  exceeding:twoihours.  „ . ^ 

Muthod  of  Moosuromont: 

Central  Control  logs. 

Passenger  Satisfaction  Surveys. 

Milosioiios; 

95%  of  alt  delays  of  3 minutes  or  more  reporled.via  station  address  system.  - 
Service  Quality  Teams  assigned  to'^all  unscheS^d  service  disruptions  deeding  2 hours.'* 

75%  of  all  passengers  returning  surveys  indicate  that  information  regarding  service  delays  is^adequate  and  useful. 

Pago  0-2 
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Id.  Development  of  annual  marketing  plan  identifying  specific  programs  and  projects  thot 
will  promote  increased  patronage. 


Goaliiic-./.'  ■ 


•Develop  Marketing  Plan  by  IMiar^lsT^'' 


Purpose: 


To  produce  a varieiy  ot  marketing  tools  that  will  provide  the  public  with  an  incer^iive  to  utilize  the  services  ot  Mum. 


Method  of  Measurement; 


Marketing  Plan  completed  and  approved  tor  implementation. 


S9|Tles{ones:.. 


lgt2DOt^MSI(’efffi?Pl?yplMTT200^^1?(ar^^ 


From  Muni  Service  Sundords  Reperi  FY 1001  end  ftlOOl 


Reported  Performance  During  the  Audit  Period 
Marketing  Plans  were  published  in  both  FY200I  and  FY2002, 

Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Muni's  Marketing  Director  provided  copies  of  the  FY200I  and 
FY2002  Marketing  Plans  as  well  as  samples  of  marketing  materials 
that  were  produced 

Ver/fJcotion  of  Reported  Performance  During  the 
Audit  Period 

Marketing  Plans  were  published  in  both  FY200I  and  FY2002. 


Problems  or  Issues  Identified  During  the  Audit  Period 

Overall, marketing  plans  were  too  ambitious  for  the  budget  provided. 
More  strategic  goals  and  objectives  should  be  set.  Including  some 
method  of  feedback  or  evaluation  to  know  that  marketing  actions 
are  achieving  expected  results.  For  example,  targeted  programs 
for  off-peak  riders  should  be  measured  by  ndership  changes  after 
program  Implementation. 

Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  has  continued  to  set  a January 
1st  milestone  for  completion  and  approval  for  Implementation  of 
the  Marketing  Plan. 


San  Francisco  Municipal  Railway  

/Action  Plan 

Budget  and  policy  planning  could  be  better  coordinated  so  that  the 
most  cost  effective  and  essential  marketing  aalvitles  are  funded.  The 
marketing  plan  identifies  activities  that  should  be  undertaken,  but 
where  there  Is  Insufficient  budget,  the  process  of  priority  setting 
becomes  ad  hoc  or  driven  by  the  opportunity  for  outside  funding. 

Key  problems  and  actions  Include: 

• Setting  priorities  for  marketing  efforts  - The 
marketing  plan  sets  out  objectives  for  Improving  Muni’s  image 
and  developing  ridership  In  targeted  markets.  Staff  has  been 
very  creative  in  developing  and  Implementing  Initiatives  to 
date,  but  with  a constrained  budget,  soff  should  set  priorities 
for  those  Initiatives  that  best  achieve  the  goal  of  increased 
patronage. 

• Utilixing  data  from  ridership  surveys  to  evaluate 
success  - There  is  no  quantitative  measurement  established 
for  this  performance  goal;  however,  data  from  ridership  surveys 
should  be  used  to  determine  whether  focused  marketing  is 
cre.ating  the  anticipated  changes  In  ridership.  For  example,  if 
the  objective  Is  to  Increase  off-peak  ridership  (on  certain  lines, 
line  segments,  or  systemwide)  then  ridership  data  collected 
could  Indicate  the  change,  and  the  Rider  Survey  could  provide 
the  reasons  for  that  change. 


PajeW 
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2d.  Publication  and  distribution  to  the  public  of  schedules  for  all  trips  taken  by  all 
vehicles  that  shall  consist  of  specific  arrival  times  at  terminals  and  established 
intermediate  points. 


Goal:  , 1 

jPublish  a complete  timetable  by  January  ).  2001 

Purpose: 

Provide  riders  with  an  updated  schedule. 

Definition  of  Measurement; 

1 

;Poblicatiqn’and  distribution  of  schedules  for  all  trips  taken  by  all  vehicles  that  shall  consist  ol  specific  arrival  timos 
<at  terminals  and  established  intermediate  points. 

Method  of  Measurement: 

Distribution  of  the  timetable  to  the  public.  Muni  is  in  the  process  of  reviewing  the  schedules  of  all  the  lines.  Onco 
the  review  is  complete,  we  will  publish  schedules  tor  individual  lines,  as  well  as  an  updated  system-wide  schedulo. 

Milestones; 

'F.Y2001  T Printing  and  distribution  of  timetable  FY2002  - Printing  and  distribution  of  timetable 

from  Muni  SwKt  StandaMi  Rtpcn,  FT  2001  and  FT7002. 


Reported  Performance  During  the>^ud/f  Period 

Muni  published  and  distributed  a complete  timetable  for  2001  on 
August  1 8.  200 1 . and  a complete  timetable  for  2002  on  August  3 1 . 
2002. 

Verificotion  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Schedules  were  developed  from  the  database  of  the  actual  runs 
bid  on  in  the  Scheduling  Department.  When  a new  driver  sign-up 
occurred,  the  Scheduling  Department  developed  the  run  cuts  and 
assignments  This  information  was  then  sent  to  the  Information 
Technology  Department  where  staff  converted  the  data  into  an 
Excel  spreadsheet  file.  This  Excel  file  was  then  sent  to  Planning 
Division  staff,  who  reviewed  and  conferred  with  Scheduling  about 
any  changes.  Schedule  changes  were  not  flagged  automatically: 
recognition  of  changes  depended  on  the  staff  person's  knowledge  of 
the  system.  Materials  were  edited  and  then  sent  to  the  publisher 


Verification  of  Reported  Performance  During  the 
Audit  Period 

Two  printed  timetables  have  been  published  during  the  audit  period 
The  first,  published  on  August  1 0.2001.  was  completed  in  .a  very  short 
time  with  the  extensive  use  of  interns.  The  second  was  published 
on  August  3 1 , 2002.  The  timetables  were  distributed  in  buses,  Metro 
stations.  City  Hall  (Info  Desk  and  thcTreasurcr's  Office)  and  the  Main 
Library.  Staff  also  filled  mail  order  requests,  and  the  Information  was 
made  available  on  the  Muni  and  regional  transportation  websites. 

Problems  or  Issues  Identified  During  the  Audit  Period 

With  the  full  implementation  of  the  new  Trapeze  scheduling  software, 
staff  will  have  the  ability  to  overlay  new  schedules  on  existing  ones 
for  a quicker  and  more  efficient  understanding  of  any  changes. 
Currently,  the  process  is  cumbersome,  time-consuming  and  reliant 
upon  the  background  and  knowledge  of  one  key  staff  person. 


Ptyt  Cr-s 
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No  new  timetable  Is  expected  to  be  published  for  2003  due  to 
the  high  expense.  Staff  Is  identifying  alternative  media  for  reaching 
riders. 

Changes  In  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  adjusted  the  milestones  to 
timetable  publications  to  the  previous  year.  The  FY2003  and  FY2004 
milestones  werc/arc; 

• FY2003  - Publish  timetable 

• FY2004  - Publish  a complete  timetable  during  FY2004 
Action  Plan 

The  auditor  identified  schedule  preparation  timeliness  and  efficiency 
as  a key  Issue  for  this  Service  Standard.  The  current  process  for 
preparation  of  the  schedules  is  cumbersome.  time»consuming 
and  reliant  upon  the  background  and  knowledge  of  one  key  staff 
person.  Timely  response  to  changes,  for  example,  those  resulting 
from  the  proposed  FY2004/2005  budget,  Is  difficult.  Ultimately, 
full  implementation  of  the  new  Trapeze  scheduling  software  will 
allow  staff  to  overlay  new  schedules  on  tixisting  ones  for  a quicker 
and  more  efficient  undersonding  of  the  changes  and  should  allow 
quicker  response.  However,  staff  must  be  adequately  trained  in  the 
use  of  the  software,  or  work  practices  will  continue  as  they  are  at 
present. 
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3d.  Operator  conduct  complaints. 


Goal:  - 

.75%  of  all'Pas’sehger  Service  Reports  vvill  beTesol?eTiT^O~days^^  feouction^Ws'sengarSefvice  ' 
Reports  annually. 

Purpose; 

Monthly  measurement  of  customer  saiisfaciion  with  the  agency  as  well  as  measuimg  (he  ettectivencss  ot 
internal  process  to  address  the  complaints. 

‘Definition  of IWeasfiremB'Aff"'' 

*i^ni  t^ill  malte  available  a summary  o^'compldlo.ttSredB^(]fresoWed  and  outstanding  oiTa  qiiatletiy  Basis. 

Method  of  Measurement; 

Data  provided  trom  the  Passenger  Service  Report  Unit  will  be  reported  to  the  Board  on  a quarterly  basis. 

MilMfdnesrrf^ ; 

voTo  vJiimn  afl  daVs  3nd“il)%“8nnu^i  reouction 

from  AluftI  Service  JlOfttfCfdj  Report  ft  2CWI  on4  FT2002. 


Reported  Performance 

In  FY200I,  90%  of  Operator  Complaints  were  resolved  within 
the  30-day  timeframe,  meeting  the  stated  goal  of  75%  resolution 
within  30  days.  In  FY2002,  only  70%  were  resolved  within  the 
timeframe,  and  the  75%  goal  was  not  met.  Performance  improved 
in  FY2003,  and  the  75%  goal  was  achieved.  Table  D-l  shows  the 
percentage  of  operator  complaints  resolved  within  30  days  (the 
number  of  complaints  closed  within  the  timeframe  divided  by  the 
number  of  complaints  recommended  for  neutral  hearings)  for 
FY2001  -FY2004, 


Table  D- 1 Percent  of  Operator  Complaints  Resolved  In  30  Days,  FY200 1 - FY2004 


FY2001 

-'FY2002 

FY2003 

1st  Quarter' 
FY2004 

^nTQoarfe'r* 
. FY2004 

ff  of  operator  complaints  requiring  tollow-up 

733 

645 

479 

93/ 

m 

# recommended  tor  neutral  hearings 

303 

264 

m 

62 

43 

closed  within  the  time  frame 

272 

199 

97 

35 

26 

% closed  within  the  time  frame 

90% 

70% 

78% 

57% 

60S 

from  Muni  Serxe  StavJar*  Quaner  fYZOO/,  mOOimOOJ  «n<j  »n  2nd  Qwner  mOOd 


OiQ  (n  (alia  Km  not  t«en  ii/ditea 


San  Francisco  Municipal  Railway 


As  shown  In  Table  D-2.  total  complaints  were  reduced  by  5% 
between  PT2000  and  Ft  1 999.  However.  Muni  did  not  attain  Its  goal 
of  a 10%  reduction  in  complaints  In  Pr200l.  FY2002.  or  Pl'2003.  In 
PT200I.  the  number  of  complaints  exceeded  the  goal  by  2,454.  In 
FY2002,  the  number  of  complaints  exceeded  the  goal  by  1 ,733.  This 
trend  continued  In  FY2003,  when  the  number  of  actual  complaints 
exceeded  that  goal  by  704.  Through  the  second  quarter  of  FY2004, 
performance  Improved.  The  actual  number  of  PSR’s  was  significantly 
lower  than  the  stated  goal. 


Table  D-2  Total  Number  of  Passenger  Service  Reports,  FY  1 999  - FY2004 


traifiQt 

FY99 

Actual 

FYOO 

Actual 

■ FYOt 
Actual 

FY01  ; 
Goal 

. ' FY02 
Actual' 

FY02" 

Goal 

Actual 

Goal 

(thrw&fi'2hd  Q) 

“PTBTGmI  I 

(through  2m)  O' 

Roduco  PSR’s  by  1U% 

12,456 

11,674 

12,931 

10,507 

13,373 

11,640 

12.740 

12.036 

5.416 

5.733\ 

From  Alunl  Stnict  Slandordi  Quoricr  rf200l,rt2002.Ff200J  oni  Quantr  FT2004. 

Dta  In  lollct  hi>«  not  b««n  audiitd 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

A detailed  process  manual  and  flow  chart  dictated  how  complaints 
were  recorded  and  resolved.  Data  were  collected  and  analyzed  in 
accordance  with  this  manual  and  the  stated  definition  and  method 
of  measurement. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Based  on  an  audit  of  data  from  Pf 200 1 and  the  first,  second,  and  third 
quarters  of  FY2002,  performance  appears  to  have  been  reported 
correaly. 

Problems  or  Issues  Identified  During  the  Audit  Period 

In  Pl'2002,  approximately  50%  of  driver  complaints  referred  for 
administrative  hearing  were  dropped  because  of  failure  of  the 
complainant  to  attend  the  hearing.  Because  of  this  situation,  a new 
procedure  for  handling  driver  complaints  went  into  effect  on  July 
i . 2003.  The  major  change  was  at  the  Division  level,  where  instead 
of  automatic  administrative  hearings,  there  is  a now  a consultation 
between  management  and  labor  as  to  the  appropriate  action,  with 
the  goal  of  empowering  management  and  looking  for  ways  to  retrain 
and  support  drivers. 

In  both  FY200I  and  FY2002,  Muni  was  unable  to  meet  its  goals  for 
reducing  the  number  of  Passenger  Service  Reports  This  will  be 
investigated  further  in  the  subsequent  task  to  develop  action  plans 
to  improve  performance. 
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Chonges  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  has  made  no  changes  to 
the  definition  of  measurement,  method  of  measurement,  nor  the 
milestones. 

Action  Plan 

This  Action  Plan  focuses  on  three  key  Issues  identified  by  the 
auditor: 

• Clarity  on  what  performance  is  being  measured 

• A need  for  realistic  goals 

• Efficiency  and  accuracy  of  complaint  information  intake 

Oarity  on  whal  Performance  is  Being  Measured 

The  operator  conduct  data  is  reported  through  the  Passenger 
Service  Report  (PSR)  database.  This  Service  Standard  is  specific  to 
operator  conduct,  not  the  whole  universe  of  PSR’s.  Measurement 
should  focus  on  the  nature  and  disposition  of  operator  conduct 
complaints;  the  FY2004  Service  Standard  Reports  are  now  more 
clearly  and  appropriately  focused  However,  the  metric  reported 
is  still  toul  PSR’s,  not  operator  conduct  complaints.  This  should  be 
revised  to  properly  reflect  the  intent  of  the  performance  standard. 

Staff  should  also  clarify  in  the  decision  flow  chart  the  criteria  and 
process  through  which  the  intake  numbers  on  operator  conduct 
complaints  are  reduced  to  non-actionable  items,  since  a large 
percentage  do  not  go  forward  due  to  such  problems  as  lack  of 
information.no  customer  identification, or  the  described  action  Is  not 
considered  an  Infraction  of  Muni  rules.  The  auditor  also  recommends 
deleting  commendations  and  station  ops  from  the  totals,  since  they 
are  not  related  to  complaints  about  operator  behavior  and  confuse 
the  understanding  of  the  goal  for  reduction. 
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It  makes  sense  to  have  the  30-day  standard  apply  only  to  the  actionable 
operator  complaints,  with  the  assumption  that  all  other  operator 
condua  complaints  have  been  suitably  disposed  of.  A reporting 
format  that  shows  total  actionable  Items  pending  and  length  of  time 
each  quarter  would  be  more  useful  than  just  showing  touts.  It  is 
not  clear  to  the  reader  whether  the  quarterly  touls  are  the  number 
received  or  the  number  pending,  The  information  system  should 
bo  able  to  generate  a report  at  any  time  {not  just  quarterly)  that 
shows  towl  pending  operator  conduct  complaints  and  time  lapsed 

for  each. 

A Need  (or  Realiiiic  Goals 

The  continued  10%  annual  reduction  in  PSR’s  Is  not  a realistic  goal 
without  some  kind  of  qualifier  such  as  a multi-year  timeframe  or 
target  threshold.  The  goal  needs  to  address  the  following  question: 
What  Is  the  acccpuble  level  of  operator  conduct  complaints  in 
the  organliation?  In  other  words,  what  would  represent  not  just 
improvement,  but  success?  One  suggestion  1$  to  look  at  the  ultimate 
urget  in  relationship  to  other  related  goals,  such  as  operator  training 
or  service  rcHablllcy.  Since,  at  Muni,  there  Is  a historical  relationship 
between  service  reliability  and  operator  conduct  complaints,  a 
realistic  urget  might  be  to  reduce  complaints  to  a ceruln  level  and 
then  not  have  them  exceed  service  reliability  indices, 

In  sotting  a base  urget,  dau  from  other  systems  could  give  an 
Indication  of  some  appropriate  levels.  For  example,  Sacramento  has 
an  average  of  3.000  toul  PSR’s  (approximately  66%  of  these  are 
operator  conduct  complaints)  quarterly  and  a daily  ridership  base 
of  100.000,  Muni  needs  to  determine  what  the  accepuble  urget 
number  of  total  PSR’s  should  be  and  the  percentage  of  those  PSR’s 


that  are  attribuuble  to  operator  condua  given  i 
congested  environment  and  a daily  ridership  of  over  700.000^  The 
undersunding  is  that  these  goals  work  together,  m that  reducing 
resolution  time  will  ultimately  result  in  changed  operator  behavior 
and  fewer  complaints. 

Eifitiency  and  Accuracy  of  Complaint  Information  Intake 

The  majority  of  the  operator  condua  complaints  are  recorded  by 
hand  during  telephone  conversations.  Other  complaints  are  received 
by  wricten/mailed  or  email  correspondence.  Personne  who  uke 
the  Information  should  be  proficient  in  dau  input  directly  onto  the 
electronic  form  as  it  is  communicated  to  them.  This  would  greatly 
Improve  efficiency  and  reduce  the  chance  of  error.  If  personnel  do 
not  have  the  skills  to  perform  these  functions,  training  shou  d be 
provided.  Improved  inuke  could  also  affect  the  number  of  complaints 
that  are  actionable  (require  follow-through). 
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4d.  Annual  passenger  surveys  and  follow-up  by  management. 


'Goal^ 

Cojiduct-a  RiderfSurvey  and  an  Emolovee  Survey; 

Purpose: 

Measure  the  level  of  satisfaction  of  transit  riders  and  employees.  Use  the  results  of  the  survey  to 
implement  improvements. 

-Definition  of  Measurement;- 

Conduct  an  annual  survey  of  riders  and  hold  focus-group  meetings  to  determine  riders' sentiments 
and  concerns.  Surveys  will  include  an  Employee  Survey  alono  with  a Ririnr  Siirupv 

Method  of  Measurement: 

Successful  completion  of  the  surveys  prior  to  the  end  of  FY2001  and  FY2002  and  present  findings 
to  Board  and  Citizens  Advisory  Committee. 

;M,ilestones: 

FY2001  - Survey  conducted  in  June  2001FY2002  - Survey  conducted  in  July  2002 

From  Munt  ServKt  Sundcrtfs  Repon.  FY 1001  ani  FtlOOl 


Reported  Performance  During  the>^ud/f  Period 

Results  of  the  FY2001  Rider  Survey  were  presented  to  the  MTA 
Board  on  August  7,  2001.  following  presentation  to  the  Citizen’s 
Advisory  Committee  (CAC).  Another  Rider  Survey  was  conducted 
in  July  2002.  Employee  surveys  were  conducted  in  June  2001  and 
June  2002-  Focus  group  meetings,  as  stated  in  the  Definition  of 
Measurement,  were  not  held. 

Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

The  Rider  and  Employee  Surveys  were  conducted  by  the  firm  of 
David  Binder  Research  under  contract  to  Muni.  Four  hundred 
respondents  aged  eighteen  or  older  who  rode  Muni  tn  the  past  six 
months  were  interviewed  on  June  1st  - 3rd,  2001  and  July  21st  - 
24th.  2002.  These  riders  were  randomly  selected  residents  of  the 
City  of  San  Francisco  with  working  telephone  numbers  that  met  the 
aforementioned  criteria.  Results  for  both  surveys  were  subject  to  a 
sampling  error  of  4.9%. 


In  June  200 1 , employee  survey  results  were  obtained  from  92 1 self* 
administered  questionnaires  from  employees  In  ten  departments, 
with  the  majority  coming  from  Transportation  and  Maintcn.ance. 
In  June  2002,  results  were  obtained  from  504  self-administered 
questionnaires  from  employees  in  1 2 departments,  with  the  majority 
again  coming  from  Transportation  and  Maintenance. 

Verification  of  Reported  Performance  During  the  Audit 
Period 

Rider  Surveys  were  conducted  in  June  200 1 and  July  2002  Employee 
surveys  were  conducted  in  June  200 1 and  June  2002. 


f^iyj^l  San  Francisco 
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Prob/ems  or  /ssues  Identified  During  the  Audit  Period 

Completion  of  the  surveys  and  communication  of  the  information 
to  the  Board  accomplishes  the  first  part  of  the  goal.  However,  the 
connection  between  findings  and  recommended  actions  for  future 
marketing  and  service  planning  initiatives  has  not  been  documented. 
In  addition,  no  protocols  have  been  established  for  trend  or  market 
segment  analysis. 


Changes  In  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  has  made  no  changes  to  the 
definition  of  measurement,  method  of  measurement,  or  milestones 
for  annual  surveys. 


Auditor  Recommendations 

In  order  to  use  the  survey  results  to  implement  improvements, 
it  will  be  necessary  to  further  develop  the  connection  between 
information,  analyses,  and  programmatic  action.  Additionally,  the 
Definition  of  Measurement  should  be  amended  to  remove  the 
sntement  “and  hold  focus-group  meetings"  unless  focus  groups  will 
actually  be  conducted. 


Pom  P-12 
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Sd.  /m^rovrements  in  public  information  regarding  vehicle  delays  during  operations  as  well 
as  any  general  user  information  regarding  system  modifications,  route  changes,  and 
schedules. 


Goal: 

Improve  Passenger  Informatior).. 

Purpose: 

Improve  all  aspects  ot  public  information. 

Definition  of  Measur'eme'iitT 

improve  passenger  imormaiion  oy  communication  of  service  problems  ano  other  intormation  to  each 
vehicle, 'the  station  platforms,  the  Telephone  informotion  Center,  media,  and  the  Service  Hotline 

Method  of  Measurement: 

Results  ot  Rider  Sunieys  will  indicate  success. 

iMilestoneSjt- 

■RlBeTSurvey-teeatiacK.  tNotrepHTteoin  FY2tf(JnrFY20D2.) 

from  Muni  StnK«  Sun^C^t  FT  2001  and  FY200Z 


Reported  Performance  During  the  Audit  Period 

This  standard  svas  not  included  in  the  Service  Standards  Reports  in 
FY200I  orFY2002, 

Veriftcotion  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Muni  did  not  report  resuits  in  this  area  in  either  FY200i  or  FY2002. 
In  fact,  there  are  no  specific  questions  on  the  Rider  Survey  that  deal 
with  customer  information.  However,  during  the  audit  period,  there 
were  a number  of  Important  changes  to  the  customer  information 
and  public  relations  functions  at  Muni  that  support  this  goal: 

• Several  different  functions  have  been  combined  under  an 
umbrella  group  responsible  for  Community  and  Public  Rela- 
tions. 

• This  department  has  developed  a "service  quality  approach 

that  puts  Muni  staff  on  site  at  locations  where  there  are  cither 
significant  changes  in  service  or  unanticipated  service  delays 
or  disruptions  to  provide  information  and  assist  passengers. 


• The  Community  and  Public  Relations  department  Is  integrated 
into  the  capital  projects  team  for  any  capiul  project  that 
is  likely  to  cause  rider  and  neighborhood  disruptions.  The 
Community  and  Public  Relations  staffs  both  provide  advice 
to  the  project  delivery  team  and  take  lead  responsibility  for 
notifications  to  the  riders  and  the  larger  public  that  may  be 
affected  by  a Muni  project 

• Muni  substantially  upgraded  its  website  and  provides  high  qual- 
ity information  regarding  service  changes,  schedule  updates 
and  disruptions. 

• During  the  audit  period,  Muni  primed  and  distributed  schedule 
books  for  all  lines  operated  by  the  railway  In  paper  format,  as 
well  as  providing  telephone  and  electronic  information. 

• The  Next  Bus  system  was  installed  In  the  subway,  providing 
real  time  information  for  subway  riders  In  addition,  a pilot 
Next  Bus  program  was  implemented  on  one  bus  route. and  is 
planned  to  be  expanded  to  the  rest  of  the  bus  system,  pend- 
ing funding. 

• Automatic  stop  announcements  on  new  vehicles  provide  use- 
ful, new  information  to  passengers  on  board  Muni  vehicles. 
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While  Muni  has  clearly  made  important  strides  In  the  area  of 
community  and  public  relations,  it  has  yet  to  adopt  a definition 
and  method  of  measurement  that  can  adequately  describe  these 
activities. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Reported  performance  could  not  be  verified  since  this  standard  was 
not  Included  In  performance  reports  in  FY200I  or  FY2002. 

Problems  or  Issues  Identified  During  the  Audit  Period 

One  of  the  reasons  this  service  standard  has  not  been  reported  on 
to  date  may  be  the  complexity  of  the  measure  itself.  The  measure 
"Improvements  In  public  information  regarding  vehicle  delays  during 
operations  as  well  as  general  user  information  regarding  system 
modifications  route  changes  and  schedules"  combines  two  very 
different  types  of  Information  - real  time  Information  that  Is  necessary 
for  customers  to  understand  and  respond  to  immediate  changes  and 
disruptions  in  the  system  and  Information  with  a longer  lead  time, 
for  construction  projects,  schedule  and  service  changes,  etc.  These 
two  distinct  types  of  Information  are  generally  not  handled  by  the 
same  people  or  the  same  departments,  which  has  made  measuring 
performance  under  this  standard  difficult. 

Muni  has  made  significant  progress  towards  Improving  customer 
information,  both  for  real  time  and  longer  lead-time  information 
needs.  Significant  efforts  have  been  made  to  develop  a clear  chain 
of  command  for  handling  information  at  the  scene  of  a major  delay 
or  service  disruption.  Muni’s  Intent  to  develop  a public  information 
plan  that  covers  all  of  these  potential  information  needs  should  be 
encouraged.  This  plan  could  be  written  In  the  format  of  a "disaster 
recovery  plan"  - Identifying  all  of  the  steps  required  to  provide 
Information  under  a variety  of  circumstances  and  who  is  responsible 
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for  managing  that  aspect  of  public  relations.  In  a well-developed 
plan,  some  responsibilities  will  lie  with  operations,  central  control, 
the  phone  room,  and  community  relations  service  teams.  Other 
responsibilities  will  belong  to  project  teams  working  on  route 
changes  and  capital  projects. 

While  Muni  has  made  substantial  progress  in  improving  customer 
information,  it  has  not  developed  methods  for  evaluating  the  impact 
of  its  information  on  customers.  For  example,  when  there  is  a major 
service  disruption  that  requires  Service  Quality  team  involvement, 
there  Is  no  follow-up  with  passengers  to  see  if  ^ey  were  satisfied 
by  the  information  they  received.  Similarly,  in  neighborhoods  where 
construction  occurs.  Muni  has  worked  diligently  to  improve  outreach, 
but  there  is  no  feedback  loop  to  determine  whether  information  was 
received  in  a timely  manner,  and  whether  it  was  helpful  to  riders. 
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Changes  in  Measurement  Since  the  Audit  Period 

The  following  revisions  to  Service  Standard  5d  were  adopted  In  FY2004; 


Purpose: 

•defmltion'of  Measure'ffi^ 


Improve  passenger  intormation  b/  communication  of  service  problems  and  other  inlormaiion  to  each  vehicle. 
Jhgjtatjon  platforms,  the  Telephone  Information  Center,  media  and  the  Service  Hotline,  and  assess. 
‘TssesScurrenf^cncf!^^^ 


from  Muni  Service  Stanrforti  eep8i\  I n Quarter  m004. 


Action  Plan 

To  be  truly  useful,  this  standard  should  be  divided  Into  two  goals 
to  distinguish  between  routine  information  provided  in  real  time 
and  longer-term  information  provided  as  part  of  service  or  schedule 
changes  and/or  construction  projects. 


The  steps  taken  to  modify  this  goal  in  FY2004  are  a good  first  step 
towards  refining  this  measure.  Muni  is  encouraged  to  complete  its 
information  plan,  providing  clear  direction  as  to  what  information 
needs  to  be  provided  and  who  is  responsible  under  a variety  of 
circumstances.  This  includes  identifying  a lead  person  for  information 
provided  on  site,  as  well  as  leads  for  individual  tasks.  These  people 
should  not  be  named,  as  individuals  change  from  time  to  time,  but  by 
position  or  department  Consideration  should  be  given  to  providing 
information  in  accessible  media  and  alternative  language  formats,  as 
Muni  serves  a diverse  customer  base. 

Muni  is  further  encouraged  to  develop  a method  of  assessing  the 
impact  of  their  information  on  the  riding  public  and  the  neighborhood 
or  community  impacted  by  their  information  campaigns.  This  could 
come  in  the  form  of  a simple  survey,  implemented  by  Service  Quality 
teams  following  incidents  or  events  where  public  information  has 
been  provided. 
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To  further  differentiate  between  immediate  information  provided 
to  passengers  on  the  system  during  delays  or  other  unusual  events, 
and  longer-term  Information  provided  to  address  construction  or 
other  longer-term  events  the  following  systems  of  measurement  are 
recommended; 


Goal: 

Improve  Real-Time  Passenger.lnformation  regarding  service  delays. . [.y 

Purpose; 

Provide  the  public  with  information  regarding  service  delays  that  will  enable  them  to  make  choices  about  their  trip. 

OefinUion  of  Moasuremont: 

Improve  passenger  inlormetion  by  communication  of 'service  problems  and  oihennioniiajidittff'each'vehlcte,  the  station':- 
platform’s,  the  Telephone  Information  Center,  media,  the  Service  Hotline;  and  Service  Quality  TeamsiOperators  vvill  make 
passenger  announcements  in  all  cases  where  service  delays  exceed  3 minutes  in  the  subway  and'10  minutes  on  surface  lines. 
Central  Control  will  make  both  visual  and  verbal  announcements  in  all  stations  where  service.delays  exceed  3 min^s  on  all 
IRV  lines.Service  Quality  Teams  will  be  sssighedio  all  unscheduled  servIce  disruptionsmeVdm^two  hours. 

Molhod  ol  Moasuromont: 

Central  Control  logs  and  Rider  or  Passenger  Surveys. 

Milostonos;  ' ' 

Plan  completed  and  implemented.--  ' ' ' -'.--i- 

PojoO-lS 


Proposition  E Municipal  Transportation  Quality  Review 

6d.  Efforts  to  improve  driver  training,  technical  as  well  as  accident  follow-up. 


Goal: _ 

J0D,000]h%s:of^rWrTrain#.pef  year  and  59rreduction  in  accidents  - .'i  . 

Purpose: 

Reduce  accidents  through  effective  operator  training  programs  as  well  as  edective  accident  lollnw-tm  ir.iininn 

iDefinition  of  Measurement;! 

1 ' 

1 

1 , 

t 

\ 

< 

j 

1 

Monthly  measurement  ol  the  number  of  training  hours  by  type  of  class.  Track  reduction  in  accidents  as  a result 
of  more  effective  operator  training  and  accident  retraining.Training  hours  will  bo  trocked  lot  the  following  areas; 
. • New.Operatof  Training 

• Immediate'FolloW’Up  Rides 

• One  Day  Accident  Retraining 
'•  Two  Day  Accident  Retraining 

• VeriRcalion’ofTransit  Training 

• Operator  Refresher 

*•  Passenger  Relations/Conflict  Training 

Method  of  Measurement: 

Wumber  of  FTA  reportable  accidents  and  training  hours.  Data  will  be  reported  to  the  board  on  a quorierly  basis 

-Milestones: 

100,000  hours  of  Oriver  Training  and  5%  accident  reduction.-' 

From  Muni  Strvict  Siandonli  Rtpert.  FT  2001  end  FY2002 


Reported  Performance 

There  were  a total  of  1 40,692  and  1 29.769  operator  trainmg  hours  in  PT200 1 and  FY2002,  respectiveiy,  exceeding  the  goai  of  1 00.000  annual 
hours  of  driver  training.  In  FY2003.  the  goal  of  50.000  training  hours  was  also  exceeded.  Table  0*3  summarizes  this  lnform.itlon.  As  shown 
in  Table  D-4.a  5%  reduction  in  accidents  was  not  achieved  in  either  FY2001  or  Pr'2002-  This  trend  appears  to  be  continuing  based  on  FY2003 
and  first  and  second  quarter  FY2004  data. 


Table  D-3  OperatorTraining  Hours,  FY200  I - FY2004 


FY2001 

FY2002 

FY2003 

Uttnd2iid 

Quarltr  2004 

.1 

Actual 

Goal 

Actual 

Goal 

Actual 

Goal 

Actual 

FYTO  Goal 

Total  Operator  Training  Hours 

140,692 

100.000 

129.769 

100,000 

82.099 

50.000 

39.705  ' 

50.000 

From  Muni  Semce  Sundanjt  f(epon.4(h  Qoartef  fY200l.fY2002,Fn002  and  In  end  2ndQuerur  fflOO* 
Otta  In  loJia  have  not  been  tjdiied. 


Table  D-4  Number  of  Accidents.  FY200 1 - FY2004 


FY2001 

FY2002 

FY2003 

Itt  and  2nd  Quartar  2004 

Actual 

Goal 

Actual 

1 Goal 

Actual 

Goal 

Actual 

FYTO  Goal 

Number  of  Accidents 

3.043 

2.6B0 

2.913 

2.891 

2.966 

2.767 

1.459 

1.409 

From  M«»w  Sevvo  Stondof*  ftepottdtft  Quofttf  FY200l,fT2002.Fr2003  end  In  end  2ndQuamr  Ff200* 
Oio  In  loSa  hxm  not  been  ludiced 
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Table  D-5  Accidents  per  100,000  Operator  Pay  Hours  Worked.  FY200 1 - FY2003 


FY2003 

Operaior'Pa'/i 
Hrs  Worked 

accidents 

Rale 

■ O'perator  Pay 
Hrs  Worked 

accidents 

P‘,'-" 

Rate.  .. 

:%change'' 

ipperatorPay, 
i.Hrs:Worked  ' 

"accidents  . 

Rate- 

l%'iihange 

Cable  car 

271,376 

116 

42.7 

292,654 

92 

31.4 

•26.5% 

125.495 

89 

70.9 

125.8i- 

Gtcen 

477,221 

424 

88.8 

478,227 

340 

71.1 

-19.9% 

583,010 

312 

53.5 

■ '-24.81- 

Flynn 

3B6.118 

458 

118.0 

410,038 

417 

101.7 

•13.8% 

369.460 

389 

105.3 

3.5‘.- 

Kirkland 

502,677 

422 

83.9 

583,531 

439 

75.2 

-10.4% 

499,m 

407 

81.5 

■ 8.4% 

PotlOfO 

562,122 

655 

116.5 

614,385 

647 

105.3 

■9.6% 

588.623 

682 

115.9 

lo.r.- 

Presidio 

470,249 

401 

85.2 

504,511 

435 

86.2 

1.2% 

483.517 

..  455 

94.1 

9.2% 

Woods 

705,383 

567 

80.3 

774,054 

543 

70.2 

■12.6% 

675.629 

632 

93.5 

33.2%  i 

Systomwido 

3,377,146 

3.043 

90.1 

3,657.400 

2,913 

79.6 

■11.7% 

3.325.112 

2.966 

89.2 

12.1% 

fi«n\  Muni  Stnict  Slsnrfofrfi  Quant/  n200l,Pf2002.FT2003. 

Oiu  In  laNu  hir«  nM  bitn  audiitd. 


In  addition  to  the  number  of  operator  training  hours  and  the  total 
number  of  accidents.  Muni  also  reported  the  number  of  accidents 
per  100,000  operator  pay  hours  worked  for  each  division.  In 
FY200I,  the  system-wide  accident  rate  per  100,000  paid  operator 
hours  worked  decreased  by  4.8%  from  94.7  in  FY2000,  to  90.1  in 
FY200I.  Between  FY200I  and  FY2002,  the  system-wide  accident 
rate  decreased  by  1 1.7%  from  90.1  to  79.6.  However,  according  to 
unaudited  FY2003  data,  the  systemwide  accident  rate  increased  by 
12.1%  between  FY2002  and  FY2003.  Table  D-5  summariies  this 
information 

Yerifjcot/on  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

The  method  of  measurement  states  that  only  FTA  reportable 
accidents  should  be  obulated.  In  fact,  all  accidents  were  counted 
in  fiscal  years  2001  and  2002.  (FTA  reportable  accidents  are  those 
accidents  that  result  In  a significant  level  of  property  damage,  injuries 
requiring  medical  attention  away  from  the  scene,  or  faolitJes.) 


All  other  information  was  collected  according  to  the  described 
definition  and  method  ofmeasurementThe  number  of  training  houn 
was  calculated  based  on  the  number  of  individuals  who  completed 
a particular  training  session  multiplied  by  the  length  (in  hours)  of 
the  training  session.  One-  and  two-day  accident  retraining  was 
not  reported  separately:  instead,  the  number  of  hours  for  one-  and 
two-day  training  were  combined  and  the  total  number  of  accident 
retraining  hours  was  reported. 

As  of  July  I,  2002.  all  accident  data  was  tracked  using  the  Transit 
Safe  Database.  Accident  information  was  entered  directly  into  the 
database  by  transit  operators  and  investigating  supervisors.  Prior 
to  this  time,  accident  reports  were  filed  in  hard  copy  and  were  then 
entered  Into  the  old  BMIS  mainframe  daobase. 
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Verification  of  Reported  Performance  During  the 
Audit  Period 

An  audit  of  operator  training  data  for  the  first  quarter  of  ^”2002 
showed  that  data  were  reported  correctly  for  this  period,  with  the 
exception  of  Violence  in  the  Workplace  training.  Violence  in  the 
Workplace  training  hours  were  double  counted  since  they  were 
counted  as  part  of  New  Operator  Training  and  were  then  counted 
again  separately.  Thus,  the  total  number  of  operator  training  hours 
forFY200l  should  be  1 38,642,  and  should  be  129,038  for  Pf'2002. 

Data  for  the  number  of  accidents  and  the  accident  rate  per  1 00,000 
operator  pay  hours  worked  were  verified  for  fiscal  years  2000, 200 1 
and  2002.  Source  data  for  operator  training  hours,  accidents,  and 
operator  pay  hours  worked  consisted  of  summary  spreadsheets 
that  had  been  compiled  by  soft  at  the  time  the  Service  Standards 
Reports  were  being  written. 

Problems  or  Issues  Identified  During  the  Audit  Period 

All  accidents  were  reported  in  FY200I  and  Ft'2002,  not  just  those 
that  were  reportable  under  FTA  guidelines.  Mum  has  already 
changed  the  method  of  measurement  for  this  indicator  to  include  alt 
accidents,  not  just  those  that  are  FTA  reportable. 

Violence  in  the  Workplace  training  hours  were  double  counted  in 
both  fiscal  years  2001  and  2002.  Violence  in  the  Workplace  training 
hours  were  counted  as  part  of  New  Operator  Training  and  were 
then  counted  again  separately.  Thus,  the  total  number  of  operator 
training  hours  for  FY2001  should  be  138,642  and  should  be  129,038 
for  FY2002. 

The  reporting  of  accident  rates  as  calculated  per  100,000  operator 
pay  hours  worked  is  not  a standard  measurement  in  the  industry. 
The  transit  industry  standard  is  to  report  accident  rates  per  miles 
driven,  hours  of  service  provided,  or  period  of  time  Furthermore, 
according  to  the  MOU  between  Muni  and  the  Transit  Workers 


Final  Report 


Union,  operator  Incentives  arc  dependent  on  the  "reduction  In  the 
tool  number  of  accidents  for  all  categories  of  revenue  accidents,” 
not  a reduction  in  the  accident  rate  per  1 00,000  operator  pay  hours 
worked.  Thus,  there  is  no  clear  reason  as  to  why  the  accident  rate 
should  be  reported  in  a manner  that  is  inconsistent  with  industry 
sandards. 

In  both  FY2001  and  FY2002,  Mum  failed  to  achieve  its  goal  of  a 
5%  reduction  in  the  number  of  accidents.  This  trend  appears  to  be 
continuing  in  FY2003. 

Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  has  adjusted  the  milestone  for 
training  and  accidents  as  follows: 

» FY2003  and  FY2004  - 50,000  hour$/S%  Accident  Reduction 

In  FY2004.  the  Method  of  Measurement  svas  .amended  to:  "Number 
of  reportable  accidents  and  training  hours.  Data  will  be  reported  to 
the  board  on  a quarterly  basis." 

Action  Pfan 

The  following  arc  relatively  minor  issues  that  should  be  addressed; 

• Violence  in  the  Workplace  training  hours  should  not  be  double 
counted  in  reports  of  operator  training  hours 

• Ensure  that  the  definition  of  me.asuremcnt  accurately  describes 
the  types  of  accidents  being  reported. 

• Consider  reporting  the  accident  rate  in  a manner  consistent 
with  transit  industry  standards,  such  as  accidents  per  miles 
driven,  hours  or  service  provided  or  over  a given  period  of 
time.  Differentiate  between  vehicle  accidents  (collisions)  and 
customer  accidents  (e  g . slips  and  falls).  Table  D*6  shows  die 
systcmvddc  accident  rate  per  1 00.000  revenue  miles  for  fiscal 
years  2001.2002  and  2003 
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Table  D-6  Accidents  per  1 00.000  Revenue  Miles. 
FY200I  - FY2003 


Revenue  Miles 

WWBBM 

24.735.581 

26,6)3,598 

28,767.205 

Numboi  ol  Accidents 

3.0^ 

2,91? 

2.966 

Accident  Rate  per  100,000 

12.30 

10.95 

10.31 

Revenue  Miles 

from  MunJ  Unkt  UoixMt  RtpvOiti  Quortc/  FY700(.  Pr2002,  FflOO). 


Dm  In  Idlict  hm  not  b>»n  |gdit«4. 


Additionally.  It  would  be  useful  to  report  types  of  accidents  according 
to  those  that  occur  during  revenue  and  non-revenue  hours,  and 
those  that  were  determined  preventable  and  non-preventabic. 

Achieving  Goals  lor  Actidenl  Reduction 

According  to  the  Office  of  Health  and  Safety  Manager,  hours  of  driver 
training  arc  not  having  a direct  impact  on  accident  numbers  and  rates, 
in  fact.  Muni  has  an  excellent  driver  training  program  that  conforms 
to  all  national  and  state  guidelines  and  standards.  Rather  than  focus 
on  driver  training,  a comprehensive  approach  and  coordinated  effon 
to  reduce  accidents  across  divisions  and  departments  is  needed.  In 
order  to  reduce  the  number  of  accidents,  safety  needs  to  be  made 
the  top  priority  throughout  the  system,  above  all  other  goals  and 
oblectlvcs.  A solid  culture  of  safety  needs  to  be  established  before 
other  performance  goals  can  be  accomplished,  othei^vise  Muni  will 
continue  to  be  unable  to  achieve  its  goals  with  regard  to  accident 
reduction. 


There  are  three  key  elements  to  reducing  accidents: 

. Engineering  improvements  (Le.,  remove  safety  problems 
through  engineering  and  technical  improvements): 

. Employing  warning  devices  and  instituting  safety  rules  and 
procedures,  including  safety  training;  and 

• Enforcing  safety  rules  and  procedures. 

Interviews  with  staff  indicate  that  Muni  is  doing  well  with  regard  to 

rhefirsttwoelements.buthas  failed  with  regardtostrictenforcemenc 

of  safety  rules  and  procedures  and  in  creating  an  organizational 
culture  that  makes  safety  the  top  priority.  Work  has  already  begun 
on  addressing  these  issues,  and  the  auditor  recommends  that  this 
continue  to  be  a focus  for  both  management  and  transit  operators 

The  number  of  accidents  and  accident  rate  is  closely  related  to 
problems  with  on-time  performance  and  headway  adherence 
The  fact  that  a significant  number  of  lines  are  running  early/sharp 
may  mean  that  drivers  are  speeding,  or  driving  in  other  ways  that 
compromise  safety.  More  street  inspectors  are  needed  to  enforce 
on-time  performance.  Run  times  and  schedules  may  need  to  be 
reevaluated  to  ensure  that  drivers  can  complete  their  routes  both 
safely  and  on  time. 
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7d  Number  of  crime  incidents  on  Municipal  Railway  vehicles  or  in  Municipal  Railway 
facilities. 


Goal:  . ‘ 

Reduction  of  5%^from  previoijs  year  • ' ■'V  ■ 

Purpose: 

To  measure  the  crime  rate  on  transit  vehicles. 

Definition  of  Measurement:^ 

? 

Quarterly,  we  reporUn  all  categories  of  crime  incidents  with  the  corresponding  quarter  for  the 
previous-year  with'a^Tchsnqe. 

Method  of  Measurement; 

Data  is  collected  by  the  Muni  Transit  Police.  Data  will  be  reoorted  to  the  bnarrl  nn  a nuaftprlw  hasic 

iMileVtones: 

5% 

From  Muni  Semct  Sto/rtfordj  Repon.  FY  2001  omj  moOl 


Reported  Performance 

The  number  of  reported  crime  incidents  includes  all  incidents  that 
occurred  on  Muni  vehicles  or  property.  Table  D-7  summarizes 
reported  performance  in  FY200I.  FY2002,  FY2003  and  the  first  and 
second  quarters  of  FY2004  with  regard  to  crime  incidents.  While 
Muni  failed  to  meet  its  goal  in  FY200I,  it  exceeded  its  FY2002  goal 
with  a reduction  of  an  additional  126  incidents.  The  FY2003  goal 
was  also  exceeded  by  the  reduction  of  an  additional  1 2 1 incidents. 


Table  D-7  Number  of  Reported  Crime  Incidents,  FY200 1 - FY2004 


FY2001 

FY2002 

FY20Q3 

1st  and  2ml  Quarter  FY2004 

1 

Actual 

Goal 

Actual 

Goal 

Actual  Goal 

Actual 

Goal 

Number  of  Incidents 

2.927  i 

2,340 

2,655 

2,781 

2.40/  2.522 

/./4B 

2.281 

From  Alum  Strvie*  SunderiJ  «epoa<th  Querur  Pr700I.FY7002.fY200J  In  Quarter  FT2004 

Oao  In  laJIcs  hive  not  been  audited. 
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Vcri/icat/on  of  Definition  and  Method  of 
Measurement  During  the  Audit  Penod 

Incident  daw  was  tracked  by  both  Muni’s  Security  Department  and 
by  the  San  Francisco  Police  Department  (SFPD).  Incident  reports 
were  generated  by  Muni  Central  Control.  Passenger  Services  and  the 
SFPD.  Those  generated  by  Central  Control  and  Passenger  Services 
were  logged  into  the  Transit  Safe  database  by  Muni  staff. 

The  SFPD  not  only  tracks  police  reports,  but  also  receives  copies  of 
Incident  reports  from  Muni.  The  SFPD  malnwined  its  own  Incident 
dawbasc,  and  provided  copies  on  a regular  basis  to  Muni  staff  so  that 
SFPD  Information  could  be  entered  into  Transit  Safe.  Muni  staff  also 
double  checked  their  Incident  records  against  those  from  the  SFPD 
to  make  sure  all  Information  was  consistent.  (Beginning  in  2004,  the 
SFPD  will  enter  incident  information  directly  into  the  Transit  Safe 
database.) 

Verification  of  Reported  Performance  During  the 
Audit  Period 

An  audit  of  daw  for  FY2002  found  that  daw  appeared  to  be  correctly 
reported  for  the  first,  third  and  fourth  quarters  of  the  fiscal  year. 
However,  there  were  slight  data  discrepancies  for  the  second  quarter, 
Swff  swicd  that  this  discrepancy  was  likely  due  to  the  fact  that  some 
additional  Incident  reports  came  In  and  were  added  to  their  tracking 
spreadsheet  after  the  Incident  daw  was  submined  for  publishing  in 
the  Service  Swndards  Report 


2nd  Quarter  FY02 


RopDrtetff|Y|  Actual 


Numboi  of  Incidonis 


676 


e»8 


More  significant  is  the  fact  that  fare  citations  issued  by  Muni  fare 
inspectors  were  not  Included  in  the  FY200I  and  FY2002  incident 
reports,  According  to  the  Security  Department  only  fare  citations 
Issued  by  the  SFPD  were  counted.  The  following  table  shows  the 
number  of  fare  ciwtions  issued  by  Muni  fare  inspectors  in  FY200I 
and  FY2002.  A towl  of  4,146  and  3.660  fare  citations  were  given  in 
FY200I  and  FY2002,  respectively.  In  comparison,  only  66  and  1 6 fare 
evasion  incidents  were  reported  in  the  FY200I  and  FY2002  Service 
Swndards  Reports,  respectively. 

Table  D-8  Number  of  Fare  Citations  Issued  by  Muni  Fare 
Inspectors 


■1^4 

FYZOOt^ 

~FY2002 

1st  Quarter 

926 

726 

2nd  Quarter 

828 

713 

3rd  Quarter 

1,522 

863 

4th  Quarter 

870 

1,358 

TOTAL 

4,146 

3,660 

Prob/ems  or  Issues  Identified  During  the  Audit  Period 

Fare  evasion  ciwtions  issued  by  Muni  fare  inspectors  were  not 
included  in  the  FY200I  and  FY2002  Service  Swndards  Reports 
Including  this  daw  Increases  the  number  of  reported  Incidents 
significantly,  but  does  not  affect  the  achievement  of  a 5%  reduction 
in  incidents  for  FY2002.  (With  the  addition  of  fare  ciwtions.  the  towl 
number  of  incidents  in  FY200 1 7,073  and  was  6.3 1 5 in  FY2002. 

This  represents  a 10.7%  reduction  in  incidents  from  FY2001  to 
FY2002.) 

The  SFPO’s  maintenance  of  a separate  incident  dawbase  has  made  it 
more  time  consuming  and  labor  intensive  for  Muni  swff  to  generate 
incident  reports.  This  issue  should  be  resolved  in  2004  when  the 
SFPD  will  begin  entering  information  dlrealy  into  the  Transit  Safe 
dawbase  as  well. 
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While  Transit  Safe  allows  staff  to  track  details  about  all  incidents,  it 
cannot  automatically  generate  some  of  the  necessary  reports.  For 
example,  total  incidents  can  only  be  tabulated  for  each  division,  rather 
than  for  the  system  as  a whole.  Additionally,  the  database  cannot 
generate  reports  about  the  total  number  of  assaults  or  arrests.  To 
tabulate  this  information,  staff  must  look  up  each  individual  case  to 
determine  whether  or  not  it  was  an  assault  or  arrest 

Coding  of  incidents  is  not  standardized  across  all  Muni  departments, 
This  sometimes  leads  to  inadequate  or  inaccurate  information  on 
incident  reports. 

Changes  In  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  has  made  no  changes  to 
the  definition  of  measurement  method  of  measurement  nor  the 
milestones. 

Auditor  Recommendations 

Fare  evasion  citations  issued  by  Muni  fare  inspectors  should  be 
included  in  the  incident  reports  included  in  the  Service  Standard 
Reports. 

A consistent  method  of  coding  crime  incidents  should  be  developed 
and  used  by  all  departments  responsible  for  collecting  this  information. 
Additionally,  new  queries  should  be  developed  within  Transit  Safe  to 
streamline  and  automate  the  reporting  of  crime  incident  data. 

The  indicator  purpose  included  in  the  Service  Standard  Report 
should  be  amended  to.'To  measure  the  crime  rate  on  transit  vehicles 
and  facilities."  This  more  accurately  reflects  both  the  content  and 
purpose  of  the  indicator. 
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E.  Employee  Satisfaction 

Employee  satisfaction  not  only  affects  staffing  performance,  but  also 
affects  customer  service  and  overall  system  performance  as  well, 
since  dissatisfied  employees  are  less  likely  to  provide  the  best  service 
possible.  Consequently,  Proposition  E included  the  following  service 
standards  aimed  at  measuring  employee  satisfaction; 

• le.  Number  of  grievances. 

• 2e.  Speed  of  resolution  of  grievances. 

• 3e.  Longevity  of  employment. 

« 4e.  Employee  recognition. 

• 5e.  Employee  education  and  training  opportunities. 

Key  Recommendations 

The  following  are  key  recommendations  based  on  the  findings  of  our 
audit  of  FY200I  and  FY2002  Service  Standards  data  and  on  trends 
observed  in  unaudited  FY2003  and  Pi'2004  Service  Standards  data: 

• In  addition  to  the  service  standards  listed  above,  Mum  is  also 
measuring  employee  satisfaction  via  its  annual  Employee  Sur- 
vey. The  results  of  this  survey  should  be  incorporated  into  this 
group  of  service  standards  since,  in  many  respects,  it  provides 
a better  measure  of  employee  satisfaction  than  the  current 
group  of  service  standards. 

• For  Service  Standard  I e.  a goal  or  benchmark  for  the  number 
and  type  of  grievances  should  be  established  so  that  manage- 
ment can  more  effectively  monitor  changes  in  grievance  rates. 
Additionally,  reporting  on  the  types  of  grievances  filed  (rather 
than  just  the  tool  number)  would  provide  valuable  informa- 
tion. 
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• For  Service  Standard  2e.  the  method  for  reporting  speed  of 
resolution  of  grievances  for  non-operator  grievances  needs  to 
be  closely  examined  and  monitored  to  ensure  that  Informa- 
tion IS  reported  correctly  in  the  future. 

• Service  Standard  3e  provides  almost  no  Information  about 
employee  satisfaction.  If  anything,  it  Is  more  a measure  of 
staffing  performance.sinco  long-term  employees  (and  relatively 
minimal  staff  turnover)  reduce  the  need  to  hire  and  train 
new  employees.  The  auditor  therefore  recommends  that  this 
measure  be  eliminated  from  the  Service  Standards  Report,  or. 
if  retained,  moved  to  the  Staffing  Performance  section.  Alter- 
natively, a more  useful  measure  of  employee  satisfaction  may 
be  the  average  length  of  service  upon  voluntary  employee 
separation. 

• For  Service  Standard  4e.  the  actual  number  and  type  of  annual 
and  quarterly  awards  made  should  be  included  in  the  Service 
Standards  Report 

• For  Service  Standard  5e.  all  employee  training  hours  should 
be  tracked  in  a centralized  system  In  order  to  facillutc  fu- 
ture reports  and  audits.  Additionally.  Muni  should  consider 
measuring  and  reporting  on  the  perceived  effectiveness  and 
usefulness  of  training  through  employee  surveys  and  training 
evaluation  forms. 
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I e.  Number  of  grievances. 


ffooP“- 
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Purpose; 

Record  and  monitor  the  status  ot  all  grievances. 

Moihod  ot  Moasuromont: 

Internal  tracking  systom  will  be  used  to  provide  data  lor  the  board  on  a quarterly  basis. 
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Reported  Performance 

This  standard  measures  grievances  filed  by  Muni  employees  and  may  include  a wide  range  of  employee  complaints.  In  FY200I,  and  the 
first  quarter  of  FY2002.  only  transit  operator  grievances  were  tracked,  while  operator  maintenance  and  other  employee  grievances  were 
monitored  for  the  last  three  quarters  of  FY2002.  Significantly  fewer  operator  grievances  were  filed  In  FY2003.  Tables  E-l  and  E-2  summarize 
the  number  of  grievances  received  and  resolved  during  the  fiscal  year,  as  well  as  those  that  remained  active  at  ^e  year’s  end.  At  the  beginning 
of  FY200I,  there  was  a backlog  of  over  300  grievances  that  were  resolved  over  the  course  of  the  fiscal  year. 


Table  E- 1 Operator  Grievances,  FY200 1 - FY2004 


rCHovanc^  S^ntTl^ 

“FV^uunffllf 

TmiomoT 

•FY2003  Total'' 

Hucoivod 

IW 

bU 

ji 

Rosolvod 

409 

1 24 

62 

t7 

Active 

9 

2 

0 

14 

lfon>  Muni  Senv*  SlandofiJi  (}uoiui  Fr200l.rfl002,F1200)  and  In  ondZodquortc/FT20(M. 

Did  In  lulici  not  bacn  tudned 


Table  E-2  Maintenance  and  Miscellaneous  Employee  Grievances.  FY200I  - FY2004 


lirluVflilcb”Slnlus" 

•mD02Totnr 

"FY2003Tot8l  ' 

Isl  nndlhd  Q FY2004 
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4 
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2 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

The  Human  Resources  Department  tracked  grievances  on  a 
quarterly  basis.  All  data  was  collected  and  reported  according  to 
the  definition  and  method  of  measurement.  The  only  exception 
that  grievances  were  not  reported  for  each  division:  instead,  the  total 
number  of  operator  and  non-operator  grievances  were  reported. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Based  on  an  audit  of  fourth  quarter  FY2002  data  for  the  number  of 
maintenance  and  miscellaneous  employee  grievances,  and  FT2002 
quarterly  data  for  operator  grievances,  data  appear  to  have  been 
collected  and  reported  accurately. 

P('200l  operator  grievance  data  could  not  be  verified  due  to  the 
fact  that  the  original  grievance  reports  no  longer  exist.  The  auditor 
did  verify  that  the  backlog  of  grievances  had  been  resolved  by  the 
end  of  FY200I. 

Problems  or  Issues  Identified  During  the  Audit  Period 

Grievances  were  not  reported  by  division,  as  described  in  the 
definition  of  measurement. 

Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  has  made  no  changes  in 
the  definition  of  measurement,  method  of  measurement,  nor 
milestones. 


Final  Report 


Auditor  Recommendations 

Either  the  definition  of  measurement  should  be  amended  so  that 
it  no  longer  states  that  grievances  will  be  tracked  by  division,  or 
the  number  of  grievances  for  each  division  should  be  reported 
separately. 

No  goal  has  been  established  for  the  number  or  type  of  grievances 
that  are  filed,  and  it  is  thus  difficult  to  use  this  measure  as  an  indication 
of  employee  satisfaction.  The  auditor  recommends  that  a goal  or 
benchmark  for  the  number  and  type  of  grievances  be  established  so 
chat  management  can  more  effectively  monitor  changes  in  grievance 
rates.  Additionally,  reporting  on  the  types  of  grievances  filed  (rather 
than  just  the  total  number)  would  provide  valuable  information. 
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2e.  Speed  of  resolution  of  grievances. 
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Reported  Performance 

Tills  standard  measures  the  speed  with  which  employee  grievances 
are  resolved.  In  FY200I  and  the  first  quarter  of  FY2002,  only 
transit  operator  grievances  were  tracked,  while  both  operator  and 
maintenance  and  other  employee  grievances  were  monitored  for 
the  last  three  quarters  of  FY2002.  However,  due  to  the  backlog  of 
over  300  grievances  at  the  beginning  of  FY200 1 , speed  of  resolution 
of  grievances  were  not  recorded  for  the  first,  second  and  third 
quarters.  Alternatively,  the  Service  Standards  Reports  described  the 
steps  that  had  been  taken  to  speed  the  resolution  of  grievances 
during  the  first,  second  and  third  quarters  of  FY200 1 . 

For  all  quarters  in  which  data  wore  reported,  75%  or  more  of 
operator  grievances  were  resolved  within  30  days.  For  maintenance/ 
miscellaneous  employees,  100%  of  grievances  were  resolved  within 
30  days  during  the  second  quarter  of  FY2002,  while  0%  and  30% 
were  resolved  within  30  days  for  the  third  and  fourth  quarters  of 
FY2002,  respectively.  In  the  fourth  quarter  of  FY2003,  97.5%  of 
operator  grievances  were  resolved  within  30  days,  while  only  33% 
of  maintcnancc/miscellancous  employee  grievances  were  resolved 
within  30  days. 


Ver/ffcat/on  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

The  timeline  for  Human  Resources'  grievance  tracking  begins  when 
an  operator  grievance  reaches  Step  3,  and  when  a maintenance/ 
miscellaneous  employee  grievance  reaches  Step  2.  Grievances  were 
assigned  a number,  and  ma}or  actions  and  associated  dates  were 
logged  by  Human  Resources  staff. 

The  time  spent  to  resolve  operator  grievances  was  based  on  when 
a Step  3 decision  was  issued.  When  time  was  waived  by  mutual 
consent  of  either  party,  the ‘‘clock  was  stopped"  and  the  waived  time 
was  not  counted.  Non-operator  grievances  were  only  considered 
to  be  resolved  if  they  were  resolved  at  the  Step  2 level.  These 
timelines  are  based  on  the  faa  chat  grievances  that  progress  beyond 
the  Step  3 (for  operators)  or  Step  2 levels  (for  non-operators)  are 
no  longer  the  primary  responsibility  of  Muni's  Human  Resources 
Department. 
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Verification  of  Reported  Performance  During  the 
Audit  Period 

An  audit  of  operator  grievance  data  for  FY2002  verified  that  ali 
data  was  collected  and  reported  correctly.  Non-operator  data  for 
FY2002  could  not  be  verified.  Human  Resources  staff  stated  diat 
they  did  not  report  speed  of  resolution  information  for  FY2002  and 
did  not  know  how  the  data  were  generated. 

Prob/ems  or  Issues  Identified  During  the  Audit  Period 

Data  on  the  speed  of  resolution  of  grievances  for  non-operators 
could  not  be  verified.  It  is  unclear  as  to  how  this  information  was 
measured. 

Changes  in  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period,  Muni  has  made  no  changes  in 
the  definition  of  measurement,  method  of  measurement,  nor 
milestones. 

Auditor  Recommendations 

The  method  for  reporting  speed  of  resolution  of  grievances  for  non- 
operator  grievances  needs  to  be  closely  examined  and  monitored 
to  ensure  that  information  is  reported  correctly  in  the  future  For 
FY2002,  this  information  appears  to  have  been  reported  incorrectly. 
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3e.  Longevity  of  employment. 
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Reported  Performance  During  the  Audit  Penod 

Tliorc  was  a significant  jump  In  longevity  of  employment  for  nearly  all  job  categories  between  FY200I  and  FY2002.  This  was  due  to  the  feet 
that  only  years  of  service  with  Muni  were  counted  In  FY200I.  and  that  years  of  service  with  the  City  of  San  Francisco  were  counted  for 
FY2002  (and  all  subsequent  fiscal  years),  It  should  be  noted  that  certain  job  classes  that  are  associated  only  with  transit  operations  are  Muni- 
specific,  and  longevity  thus  reflects  only  years  of  service  with  Muni.  Table  E-3  summarizes  employee  longevity  for  these  two  years. 


Tabic  E-3  Employee  Longevity,  FY200 1 - FY2002 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

The  Position  Control  System  (PCS)  database  was  used  to  track  all 
Muni  personnel  data,  including  changes  in  appointment,  temporary 
and  permanent  separations  (by  type),  etc.  The  PCS  database  was 
used  to  generate  a report  on  the  average  years  of  Muni  service  in 
each  job  class  and  category. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

Based  on  an  audit  of  FY200 1 and  FY2002  data,  all  data  on  employee 
longevity  appears  to  have  been  reported  accurately. 

Problems  or  Issues  Identified  During  the  Audit  Period 

Only  years  of  service  with  Muni  were  reported  in  FY2001,  while 
total  years  of  service  with  the  City  of  San  Francisco  were  reported 
for  all  non-Muni  specific  job  categories  in  FY2002. 

Chonges  In  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  has  made  no  changes  in 
the  definition  of  measurement,  method  of  measurement,  nor 
milestones. 
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Auditor  Recommendations 

The  Service  Standards  Reports  $how“Average Years  of  Muni  Service," 
This  statement  is  incorrect,  since  the  longevity  report  actually  shows 
average  years  of  service  with  the  City  of  San  Francisco  for  ali  job 
categories  that  are  not  Muni-specific  (e,g..Transit  Operators).  Either 
this  label  should  be  changed  to  accurately  reflect  the  information 
that  is  reported,  or  only  years  of  Muni  service  shouid  be  calculated 
and  reported. 

This  Service  Standard  provides  almost  no  information  about 
employee  satisfaction.  If  anything,  it  is  more  a measure  of  staffing 
performance,  since  long-term  employees  (and  relatively  minimal 
soff  turnover)  reduce  the  need  to  hire  and  train  new  employees. 
The  auditor  therefore  recommends  that  this  measure  be  eliminated 
from  the  Service  Standards  Report,  or,  if  retained,  moved  to  the 
Staffing  Performance  section.  Alternatively,  a more  useful  measure 
of  employee  satisfaction  may  be  the  average  length  of  service  upon 
voluntary  employee  separation. 
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4e.  Employee  recognition. 


Gonl:  ■'Y  1 

!*  ! 

Annual  achievemant  ofthonoraes  in  i[w,Jollowing;programsJ-^j8§2!^^-s‘^^^  j 

. (121.V Systemwide  Op^ators  of  the'Mbnth  ^ard  -1 

U>— 141  'Transit  Sup’ervisor'sof  the  Quarter'AWsrd-  ' . jJ-- 

• (4)  t^Finance  & Admin  Emplo^'Of  the^Q^gen  - J]'' 

. • |121  Maintenance  Employees  of  the  Montn^'’  ^ ■e*»‘ • , 

' •'  {4),^JVl3fnTenancc,SuE8rvisors'oftlie  Qu  ’ , - - ^4 

'•  (4)' ’Safety &Trai^ni)Etnployee  of  the  Quarter 

' . Ml.yMii^VtihllltiiiFfnnlnvpanf thRDiiarter-'  ^ - v.^:^ 

Purposo: 

To  reconnire  the  achievements  of  employees  and  encoutape  excellence  in  )ob  pertormance.  ._uu..,  — 

Dotinitian  of  Monsuromonf. 

: , ■ - , 

Monthly  tracking-of  all-award  piogiams.  Awatd-’progfam  criteria  vary  for^^ahow^detail^^^ 
process  exists  on  file  for  aach  pmgi^arrOTrlteria ^on-operator  avvards  includes,  ^s  not  hmited  to  employee  ^ •/ 
PerformBnce’evoluation^  at^ilance,*work  performance; absence.of  disciptinar^and  or  EEQ^easures..  For^O^ratw  . 

.awards;  ottefiiloTe  lecb®  ccident  records,  PSR's  and  safety  records  are'used.lo  evaluate  the.candlifite.  » 

Molhod  of  Moasuromont: 

A doloiled  nomination  evaluation  process  exists  on  file  for  each  program  and  the  time  frame  measured  is  generally 
on  a rolling  12  month  basis,  Some  of  the  criteria  includes  employee's  performance  evaluation,  attendance,  work 

Mllostonos; 

Annual  achievement  - 

from  AtunJ  J*»v4te  fY  2001  «nd  fY200l 
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Reported  Performance 

There  was  no  reported  performance  for  this  indicator  in  either  the 
FY200I  or  FY2002  Service  Standards  Reports. 

Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

The  definition  and  method  of  measurement  could  not  be  verified. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

The  following  awards  were  made  during  calendar  year  200 1 : 

• (10)  Systemwide  Operators  of  the  Month  Award 

• (2)  Transit  Supervisor's  of  the  Quarter  Award 

• (3)  Finance  & Admin  Employee's  of  the  Quarter 

• (IS)  Maintenance  Employees  of  the  Month 

• (0)  Maintenance  Supervisors  of  the  Quarter 

• (2)  Safety  & Training  Employee  of  the  Quarter 

• (0)  Accessibility  Employee  of  the  Quarter 

Additional  awards  were  made  for  Construction  Division  Employees 
of  the  Quarter  (2),  Capital  Planning  and  Legislative  Affairs  Division 
Employee  of  the  Quarter  (3),  and  Human  Resources.  Labor  Relations, 
and  EEO  Division  Employee  of  the  Quarter  (3). 

All  honorees  received  formal  recognition  at  an  awards  banquet, 
gift  certificates,  and  were  eligible  to  win  major  prizes  All  those 
nominated  for  awards  were  recognized  as  well. 
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Problems  or  /ssues  Identified  During  the  Audit  Period 

The  Service  Standards  Reports  did  not  report  on  the  actual  number 
and  type  of  awards  made  during  each  fiscal  year,  although  some  of 
the  goals  were  mcL 
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Se.  Employee  education  and  training  opportunities. 
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I enn.in„m,;oDOort»nilies  ior  employee  developmeni  ^ 

Ttaininfl  hours  will  be  locked  monthly  lor  u^lowmg 
i-  . MointDronce Ttami«B-ISndt«lini! nnuJ^B-vcbicte traininB) 

^ '•  7Habit1'ofHighlvEHective People 

• Ambassador  Training  . . 

• Supervisory  Skills  Training  ^ v. 

• Managemenl Skills*Ttaining  ’ ' • '■-•  •....- 

• Violence  in  the  Workplace 

• Oesktop^fcbmputer  Training 

. . ..  k ■ . • * — Ja.1 1 Ain Anni 
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Chonges  In  Measurement  Since  the  Audit  Period 

Subsequent  to  the  eudit  period,  Muni  hss  mode  no  chenges  l^n 
the  definition  of  mensuroment,  method  of  measurement,  no 

milestones. 

Auditor  Recommendatroris 

The  ectusi  number  end  type  of  ennusi  end  quarterly  swords  mode 
should  be  Included  in  the  Service  Standards  Report. 


Reported  Performance 

This  stonderd  measures  training  hours  for 

FY200 1 53.296  hours  of  training  v^ere  provided,  and  50  880  hour 
^f^llg  were  provided  in  FY2002.  In  FY2003.  31.241  hours  of 
training  were  provided  (less  than  the  goal  of  50.000  annua  ^ming 
hour^)i  and  7. 1 69  hours  were  offered  in  the  first  quarter  of  FY2  ^ 
Training  was  provided  in  all  of  the  categories 
definition  of  measurement  with  the  exception  of  the  J of 

Highly  Effective  People"  training  which  was  not  provided  in  FY2002 

and  FY2003, 
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Verification  of  Definition  and  Method  of 
Measurement  During  the  Audit  Period 

Maintenance  and  administrative  training  were  tracked  in  a number 
of  ways.  Administrative  training  includes  regular  in-house  training 
sessions  as  well  as  conferences,  retreats,  and  seminars  attended  by 
staff  members. 

For  in-house  training,  the  number  of  people  who  completed  each 
class  was  recorded  and  the  total  number  of  training  hours  was 
computed  by  multiplying  the  number  of  attendees  by  the  number  of 
class  hours.  For  external  conferences,  seminars,  etc,,  staff  members 
who  attended  these  events  were  asked  to  report  the  number  of 
hours  spent  at  each  event. 

Verification  of  Reported  Performance  During  the 
Audit  Period 

An  audit  of  FY2002  reported  training  hours  for  Violence  in  the 
Workplace  training  found  that  the  number  of  training  hours  was 
correaly  reported  for  all  quarters  except  for  the  fourth.  For  the 
fourth  quarter  of  FY2002,  there  were  261  training  hours  reported 
in  the  Service  Standards  Report.  Data  collected  by  the  Training 
Manager  showed  that  34S  training  hours  were  administered  during 
the  fourth  quarter  of  FY2002. 

An  audit  of  FY2002  third  and  fourth  quarter  reported  training  hours 
for  Drug  and  Alcohol  training  verified  that  supervisor  Drug  and 
Alcohol  training  hours  were  reported  correctly  for  both  quarters, 
and  employee  Drug  and  Alcohol  training  was  reported  correctly  for 
the  fourth  quarter.  For  the  third  quarter,  a total  of  264  employee 
Drug  and  Alcohol  training  hours  were  reported.  However,  according 
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to  the  Training  Supervisor,  3 1 6 hours  of  employee  Drug  and  Alcohol 
training  were  administered  during  the  third  quarter  of  FY2002. 

Class  logs  were  provided  for  Maintenance  Training  for  FY200I  and 
FY2002,  however  not  enough  information  was  provided  to  derive 
the  reported  number  of  hours  of  Maintenance  Training  included 
In  the  Service  Standards  Reports  For  other  types  of  training 
hours,  records  were  incomplete  or  nonexistent  and  the  reported 
performance  could  not  be  verified. 

Problems  or  Issues  Identified  During  the  Audit  Period 

No  significant  Issues  or  problems  were  identified.  However,  some 
type  of  centralized  database  or  record  keeping  system  that  could 
be  used  to  keep  track  of  all  training  hours  would  faciliuto  future 
reporting  and  audits. 

Changes  In  Measurement  Since  the  Audit  Period 

Subsequent  to  the  audit  period.  Muni  has  made  no  changes  In 
the  definition  of  measurement,  method  of  measurement,  nor 
milestones. 

Auditor  Recommendations 

All  employee  training  hours  should  be  tracked  In  a centralized 
system  in  order  to  facilitate  future  reports  and  audits.  Additionally, 
Muni  should  consider  measuring  and  reporting  on  the  perceived 
effectiveness  and  usefulness  of  training  through  employee  surveys 
and  training  evaluation  forms. 
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Conclusion 

Table  F-l  summarizes  recommendations  for  each  service  sundard. 


Table  F.  I : Sumrr>ary  of  Auditor  Recommendations 


RKommuditions 

la.  % ot  vehicles  that  run  on  time  according  to 
published  schedules. 

• Reduce  incidence  of  early/sharp  operations. 

Schedule  evaluationlremediation  to  improve  on-time  performance. 

• Route  evaluation  to  improve  on-time  performance. 

• Targeted  improvements  directed  at  speciPc,  under-performing  lines. 

• Reallocation  of  terminal  checking  to  increased  NTO  checking  (after  improvement  of  oafly/sharp  operation 
IS  achieved). 

• IHnf  schedule  adherence  only  for  those  transit  services  with  heedway  intervals  of  mote  than  10  min 
(Will  require  e City  Charter  amendment  and  may  not  be  feasible  in  the  near  term.) 

• Coordinate  la  qoal  with  other  ooals  anri  nhiartiwpt  narttpuioftK  thnm  n-,  e. 

2a.  % of  scheduled  service  hours  that  are  delivered 
and  % of  scheduled  vehicles  that  begin  senrice  at 
the  scheduled  time. 

• Develop  a plan  for  maximizing  service  provision  during  “low  availability  periods.” 

3a.  Missed  service  due  to  either  insufficient  vehicles 
Of  driver  unavailability  as  a percentage  of 
scheduled  service  hours. 

• Eliminate  this  service  standard  since  it  is  simply  an  inverse  of  2a. 

4a.  % of  vehicles  that  pass  published  time  points 
during  measurement  periods  unable  to  pick 
up  passengers  due  to  crowding  without  being 
followed  within  3 minutes  or  less  by  another 
vehicle  on  the  same  route  with  space  for  all 
waiting  passengers. 

• Include  a systemwide  pass-up  table  by  quarter  with  boih  number  and  S;  show  previous  quarters  and 
present  an  annual  summary. 

• Track  individual  pass-up  line  performance  over  each  quarter  of  each  fiscal  year. 

• Coordinate  with  5a  so  that  all  lines  identified  as  exceeding  85S  capacity  during  peak  periods  oro 
monitored  as  part  of  4a. 

• Additional  data  available  through  the  Passenger  Service  Reports  database  should  be  used  to  conduct 
follow-up  traffic  checks  on  lines  chronically  identified  as  having  pass-ups  or  peak  period  loads  exceodmo 
85S  of  capacity. 

PajaF-i 
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Service  Standard  Recommendatiens 


Service  Standard 

Recommendations 

So.  Paok  period  passenger  bed  factors. 

• For  noncompliant  lines,  sampling  frequency  should  be  increased  until  performance  improves. 

• May  be  more  meaningful  to  capture  individual  line  load  factors  by  15  min  periods  rather  than  a for  a full 
hour. 

• Coordinate  with  4a  so  that  all  lines  identified  as  exceeding  85%  capacity  during  peak  periods  are 
monitored  as  part  of  4a. 

• Additional  data  available  through  the  Passenger  Service  Reports  database  should  be  used  to  conduct 
follow'up  traffic  checks  on  lines  chronically  identified  as  having  pass-ups  or  peak  period  loads  exceeding 
65%  of  capacity. 

6o.  Actuol  hoodways  ogoinst  scheduled  headways 
on  nil  radial,  express,  cross-town,  secondary  and 
feeder  linos  for  ponk,  base,  evening  and  late  night 
sotvicos. 

• Measure  headway  adherence  only  for  those  transit  services  with  headway  intervals  of  10  min  or  less. 
(Will  require  a City  Charter  amendment  and  may  not  be  feasible  in  the  near  term.) 

• Schedule  evaluationlremediation  to  improve  headway  adherence. 

• Route  evaluation  to  improve  headway  adherence. 

• Reallocation  of  terminal  checking  to  increased  NTO  checking  (after  improvement  of  earlylsharp  operation 
is  achieved). 

• Coordinate  6a  goal  with  other  goals  and  objectives,  particularly  those  for  la.  2a  and  6d. 

• Change  the  defnition  of  headway  adherence. 

7o.  % vohicio  availability  end  reliability  (mean 
distance  botwoon  foilurol  by  mode. 

• Consider  reporting  information  on  spare  ratios  to  determine  underlying  causes  of  vehicle  availability 
problems. 

• Correct  definition  of  measurement  to  remove  the  statement.  ‘An  additional  measurement  is  added  which 
shows  the  number  of  days  when  a facility  (mode)  does  not  have  100%  availability,*  or  include  this 
Information  in  the  Service  Standards  Report. 

8o.  Unscheduled  absences  by  operator,  mechanical 
ond  administrative  personnel. 

• Consider  establishing  a more  reasonable  goal  for  the  % of  unscheduled  absences  in  the  Administration 
Division. 

So.  Milos  botwoon  toadcolls  by  mode. 

• Calculate  quarterly  M06F  in  a consistent  manner. 

IB.  SYSTEMlfiERFORWAK^”  ' 

- - . . . _ 1 

lb.  Passongors  carried  by  mode. 

* Remove  ‘Data  will  be  reported  to  the  board  on  a quarterly  basis'  from  the  Definition  of  Measurement. 

* Realistic  goals  for  ridership  should  be  established  based  on  economic  conditions  as  well  as  available 
services. 

2b.  Faro  rovenuos  genotoicd  by  mode. 

• Calculate  average  tare  per  passenger. 

* Change  milestone  to:  'Fare  revenue  should  increase  by  1.5%  per  year  compared  with  prior  year 
performance.* 
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1 Service 

3b.  Hours  and  miles  operated  by  mode. 
4b.  Expenses  incurred  by  mode. 

■ Miles  and  hours  of  service  are  a direct  reflection  of  availablTSTSTSS 
standard  (data  will  still  be  reoorted  for  NTD  purposes) 

• Revise  the  performance  measurement  system  to  track  fully  allocated  co.sts  per  hnur  nt  hy  -T,«d‘r 

"r^fiifiTnuifiii  1 1 II  npipwi 

1c.  Net  vacancies  by  position  (vacancies  remaining 
once  promotions  and  new  hires  have  been 
deducted  from  retirees  or  resignations!  for  each 
division. 

• Limit  measurement  to  Transit  Operators  and  Maintenance  personnel. 

2c.  Attrition  rates  for  new  employees,  by  division  and 
level. 

• Data  do  not  provide  information  on  the  attrition  rates  for  new  employees.  To  calculate  attrition,  only 
those  employees  who  separate  after  six  months  or  one  year  of  service  should  be  counted. 

^ • If  possible,  use  available  information  to  report  on  reason.s  for  i/nlimtarv  <pnflrntinn< 

Id.  Development  of  annual  marketing  plan  identifying 
specific  programs  and  projects  that  will  promote 
increased  patronage. 

• Set  priorities  for  marketing  efforts, 

• Utilize  data  from  ridership  surveys  to  evaluate  success. 

2d.  Publication  and  distribution  to  the  public  of 
schedules  for  all  trips  taken  by  all  vehicles  that 
shall  consist  of  specific  arrival  times  at  terminals 
and  established  intermediate  points. 

• r-uiiy  utilize  capabilities  of  new  Trapeze  software  to  simplify  and  streamline  publishing  of  timotobles. 
Ensure  that  staff  have  adequate  training  in  use  of  the  new  software. 

3d.  Operator  conduct  complaints. 

• Clarify  what  performance  is  being  measured. 

• Set  realistic  goals  for  reducing  operator  conduct  complaints. 

• Improve  the  efficiency  and  accuracy  of  complaint  information  uptakf 

4d.  Annual  passenger  surveys  and  follow-up  by 
management. 

• In  order  to  use  survey  results  to  implement  improvements,  it  will  be  necessary  to  further  develop  the 
connection  between  information,  analyses,  and  programmatic  action. 

• Amend  Definition  of  Measurement  to  remove  focus  groups 

5d.  Improvements  in  public  information  regarding 
vehicle  delays  during  operations  as  well  as 
any  general  user  information  regarding  system 
modifications,  route  changes  and  schedules. 

Further  revise  the  Goal.  Purpose,  Definition  and  Method  of  Measurement,  and  Milestones  lor  this  service 
standard. 
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Serviefl  Slandird 

Recommendations 

6d.  Efforts  10  improve  driver  troining,  technical  as 
woil  os  accident  follow  up. 

• Do  not  double  count  Violence  in  the  Workplace  training. 

• Ensure  that  Definition  of  Measurement  accurately  describes  the  accidents  being  measured. 

• Consider  reporting  the  accident  rate  in  a manner  consistent  with  industry  standards  (e.g.,  accidents  per 
100,000  revenue  miles). 

• Implement  a comprehensive  approach  and  coordinated  effort  to  reduce  accidents  across  divisions  and 
depariments  by  focusing  on  enforcement  of  safety  rules  and  procedures  and  thorough  review  of  all 
accidents. 

• Report  types  of  accidents  by  revenue  vs.  non-revenue  hours  and  preventable  vs.  nen-preventabla. 

7d.  Number  of  crime  incidents  on  Municipal  Railway 
vohiclos  or  in  Municipal  Railway  facilities. 

> Include  fare  evasions  issued  by  Muni  fare  inspectors. 

• Develop  a consistent  method  for  coding  crime  incidents. 

• Amend  indicator  ouroose  to  *70  measure  the  crime  rate  on  transit  vehicles  and  facilities." 

IE.  EMPLOYEE  SATISFACTION  . 

j Tv;:,.-.  jiP'---'  ^ 

1o.  Number  of  griovoncos. 

* Grievances  are  not  tracked  by  division.  Either  amend  the  Definition  of  Measurement  or  change  reponing. 

* Establish  a goal  or  benchmark  for  the  number  of  grievances  filed. 

* Report  on  types  of  grievances  tiled. 

2o.  Speed  of  resolution  of  grievances. 

* Closely  examine  method  for  reporting  speed  of  resolution  of  non-operator  grievances  to  ensure  that  data 
are  reported  correctly  in  the  future. 

3o.  Imigovity  of  employment. 

• The  Service  Standards  Reports  show  "Average  Years  of  Muni  Service.'  This  terminology  is  incorrect 
since  the  report  shows  average  years  of  service  with  the  City  of  San  Francisco  for  ail  job  categories  that 
ate  not  Muni-specific. 

• This  measure  does  not  provide  any  useful  information  on  employee  satisfaction.  Either  eliminate  this 
service  standard,  or  move  it  to  the  Staffing  Performance  section.  Alternatively,  measure  the  average 
length  of  service  upon  voluntary  employee  separation. 

4o.  Employoo  recognition. 

' Report  on  the  actual  number  and  type  ol  awards  made  quarterly  and  annually. 

5o.  Employoo  oducollon  ond  training  opportunities. 

• Track  all  employee  training  hours  in  a centralized  system  in  order  to  facilitate  future  reports  and  audits. 

* Consider  measuring  end  reporting  on  the  perceived  effectiveness  and  usefulness  of  training  through 
employee  surveys  and  training  evaluation  forms. 
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Proposition  E Service  Standards  - Staff  and  Auditor  Assignments 


Service  Standard  Number 

Performance  Indicator  Name 

Responsible  Individual  (a) 

NelsonlNyBaard 
Team  Auditor 

ia.  SYSTEM  RELIASIIITY 

la.  SYSTEM  RELIABILITY 

% of  Vehicles  that  run  on  time  according  to  published  schedules  (no 
more  than  4 minutes  late  or  1 minute  early)  measured  at  terminals  end 
established  intermediate  points. 

Susan  Chelone 
SCHEDULES 

Angolo  Figono 

2a.  SYSTEM  RELIABILITY 

% of  Scheduled  service  hours  that  are  delivered  and  percentage  of 
scheduled  vehicles  that  begin  sen/ice  at  the  scheduled  time. 

rv - - 

1 ctcr  ucf 

opEfwneNs 
For  FY04-Ed  Wong 

Angolo  Figono 

3a.  SYSTEM  RELIABILITY 

Missed  service  due  to  either  insufficient  vehicles  or  driver  unavailebility  as 
a percentage  of  scheduled  service  hours 

Peter-Ber 

riCroAIinilC 
ur  tfiWttUrrO 

ForPY04-Ed  Wong 

Angolo  Figono 

4a.  SYSTEM  RELIABILITY 

% of  Pass-ups  determined  when  supervisory  personnel  review  all  the  lines 
checked  in  the  previous  quarter,  and  identify  the  five  lines  with  the  highest 
load  factors,  and  the  time  period  these  load  factors  occurred. 

Susan  Chelone 
SCHEDULES 

Angolo  Figono 

5a.  SYSTEM  RELIABILITY 

Peak  period  passenger  load  factors 

Susan  Chelone 
SCHEDULES 

Angolo  Figono 

6a.  SYSTEM  RELIABILITY 

Actual  headways  against  scheduled  headways  on  all  radial  express,  cross- 
town, secondary,  and  feeder  lines  lor  peak,  base,  evening,  and  late  night 
services. 

Susan  Chelone 
SCHEDULES 

Angolo  Figono 

7a.  SYSTEM  RELIABILITY 

% vehicle  availability  by  mode. 

Deborah  Denison 
OPERATIONS’ADMIN 

Kara  Vulcich 

8a.  SYSTEM  RELIABILITY 

Unscheduled  absences  by  operator,  mechanical  and  administrative 
personnel. 

Deborah  Denison 
Carolyn  Lew  IT 

Kara  Vuicich 

9a,  SYSTEM  RELIABILITY 

Miles  between  roadcalls  by  mode 

1 

Jack  Hieda-LRV 
Virgil  Dennis-Motor  Coach 
Woon  Wong-Piesidio  Trolley 
Tony  Chan  - Potrero  Trolley 
Patrick  Ho  - Cable  Car 

Kara  Vuicich 
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Service  Slendard  Number 

Performance  Indicator  Name  Responsible  Individual  (s) 

NelsonlNygaard  1 
Team  Auditor  1 

ii.  SYSTEM  PERFOnMANCEiHi 
1b.  SYSTEM  PERFORMANCE* 

Passongors  carried  by  mode  - FTA  submittal  • audited 

SchetItilM 
Susan  Chelone 

Kara  Vuicich 

2b.  SYSTEM  PERFORMANCE 

Fore  revenues  generated  by  mode?  FTA  submittal  • audited 

Eloida  Leonardo 
Finance 

Bonnie  Nelson 

3b.  SYSTEM  PERFORMANCE* 

Hours  and  Milos  operated  by  mode  • FTA  submittal  ■ audited 

Bavid  Makelsky 
Susan  Chelone 

Kara  Vuicich 

<lb.  SYSTEM  PERFORMANCE 

Expenses  incurred  by  mode 

Fred  Clarke 
Finance 

Bonnie  Nelson 

,c.  STAFFING  PERFORMANCE 
Ic.  STAFFING  PERFORMANCE 

Net  vacancies  by  position  (vacancies  remaining  once  promotions  and  new 
hires  have  been  deducted  (rom  retirees  or  resifinations)  for  each  division. 

Lynn  Prince 
HR 

Kara  Vuicich 

2c.  STAFFING  PERFORMANCE 

Attrition  rates  lor  now  employees,  by  division  and  level 

Lynn  Prince 
HR 

Kara  Vuicich 

d.  CUSTOMER  SERVICE 
Id.  CUSTOMER  SERVICE 

Oeveiopmenl  ol  an  annual  marketing  plan  identifying  specific  programs 
and  ptoiecis  that  will  promote  increased  patronage. 

Mark  Caposino 
Marketing 

Carmen  Clark 

2d,  CUSTOMER  SERVICE 

Publication  and  distribution  to  the  public  of  schedules  for  all  trips  taken  by 
all  vehicles  which  shall  consist  of  specific  arrival  times  at  terminals  and 
osteblished  inlermediata  points. 

Alan  Siegel 
Community  Affairs 
Bob  Calwell 
Planning 

Carmen  Clark 

3d.  CUSTOMER  SERVICE 

Operotor  conduct  complaints  and  their  resolution,  by  complaint,  consistent 
with  duo  process  and  required  conlidentiality. 

Maria  Williams 
PSR  Unit 

Carmen  Clark 

4d,  CUSTOMER  SERVICE 

Annual  possenget  surveys  and  lollow  up  by  management. 

Mark  Caposino 
Marketing 

Carmen  Clark 

5d.  CUSTOMER  SERVICE 

Improvements  in  public  information  regarding  vehicle  delays  during 
opomtions  os  well  os  gonerol  user  information  regarding  system 
modifications,  route  changes,  and  schedules. 

Jeff  Hamilton 
Community  and  Public 
Relations 

Bonnie  Nelson 

Proposition  E Municipal  Transportation  Quality  Review 


Final  Report 


^ 

1 Service  Standard  Number 

PerformancR  IniiicatAr  Uama 

NsUonlNygaard 

6d.  CUSTOMER  SERVICE 

tlfons  to  improve  driver  training,  technical  as  well  as  accident  follow-up. 
* Ron  Hundenski  provided  most  of  the  accident  data  in  the  audit 
time  period.  Now  provided  by  Paul  Lovette 
**  Bud  McNaughton  retired.  Currently  provided  by  Doris  Lanier 

Rofr-Htmdenski 
PotjHoveife 
Occidents 
Brnf-McNaughton- 
*~Tfammg 
Mike  Kirchanski 
Doris  Lanier 
Paul  Petersen 

nim  Auditor 

7d.  CUSTOMER  SERVICE 

le.  EMPLOYEEiSATrSFACTlONr’ 

Number  of  crime  incidents  on  Municipal  Railway  vehicles  or  in  Municipal 
Railway  facilities. 

Michelle  Blalock 
Security 

Kara  Vuicich 

le.  EMPLOYEE  SATISFACTION 

Number  of  grievances 

Mike  Casey 
Human  Resources 
Mabel  Shaw 
Human  Resources 

Kara  Vuicich 

2e.  EMPLOYEE  SATISFACTION 

Speed  of  resolution  of  grievances 

Mike  Casey 
Human  Resources 

Kara  Vuicich 

3e.  EMPLOYEE  SATISFACTION 

Longevity  of  employment 

MfthaeHiim 
Human  Resoufces 
Hannah  Staroselsky 
Human  Resources 

Karo  Vuicich 

4e.  EMPLOYEE  SATISFACTION 

Employee  Recognition 

Employee  Recognition 
Program  Oiganirer 
Orange  Rose 

Kara  Vuicich 

5e.  EMPLOYEE  SATISFACTION 

Employee  education  and  training  programs 

Deborah  Denison 
Vernon  Crawley 
flith-Oate 
Patty  Deleon 
Barbara  Convray 
Paul  Petersen  I 

Kara  Vuicich 
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